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ABSTRACT 


The Navy spends over $300 million per year to recruit approximately 
35,000 new active duty enlisted Sailors. The Navy has historically used a non¬ 
linear optimization model, the Planned Resource Optimization (PRO) model, to 
help inform decisions on the allocation of those recruiting resources. Input 
variables to the PRO model include economic influences and policy factors. The 
result is a recommended allocation of resources for advertisements, recruiters, 
enlistment bonuses, and education incentives. The PRO model’s primary 
limitations are (1) potential deviations of input variables are not taken into 
consideration, and (2) extensive experimentation is not feasible. Realistically, 
input variables to the PRO model fluctuate, are unpredictable, and can interact 
with other variables to influence the recruiting environment and affect the optimal 
allocation of recruiting resources. This paper describes the “Planned Resource 
Optimization Model with Experimental Design” (PROM-WED), a tool that 
alleviates the limitations and enhances the analytic utility of the legacy PRO 
model. PROM-WED embeds the legacy PRO model within a data farming 
environment. PROM-WED’s graphical user interface and decision support 
capability provide decision makers with robust insights into variable interactions 
and uncertainties to better inform their recruiting resourcing decisions. 
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EXECUTIVE SUMMARY 


The mission of the United States Navy is “to maintain, train and equip 
combat-ready Naval forces capable of winning wars, deterring aggression and 
maintaining freedom of the seas” (United States Navy, n.d.-b). Congress 
authorizes the Navy to maintain a force-strength of over 300,000 active duty 
personnel to execute this mission (Government Accountability Office, 2016, p. 5). 
The Navy spends over $300 million to recruit approximately 35,000 new active 
duty enlisted Sailors each year to sustain its manning strength (Department of 
the Navy, 2015, p. 7). Under the Deputy Chief of Naval Operations for 
Manpower, Personnel, Training and Education (N1/MPT&E), analysts use 
mathematical models to support decision makers on personnel and budget 
related resourcing issues (United States Navy, n.d.-a). One model that N1 has 
historically used to inform recruiting resourcing decisions is the Planned 
Resource Optimization (PRO) model. 

The PRO model is a deterministic non-linear optimization model that 
provides users with a recommended set of resources to minimize the cost of 
Navy recruiting (Green & Mavor, 1994). The PRO model optimizes resources 
allocated to advertisements, enlistment bonuses, education incentives, and 
recruiters. Input variables to the model include economic influences such as 
unemployment rate and policy factors such as percentage of high quality recruits 
(Navy Recruiting Command, 2007). Limitations of the PRO model are (1) potential 
deviations of input variables are not taken into consideration, and (2) extensive 
experimentation is not feasible. Realistically, the input variables to the PRO 
model fluctuate, are unpredictable, and can interact with other variables to 
influence the recruiting environment and affect the optimal allocation of recruiting 
resources. 

To alleviate the limitations and enhance the analytic utility of the legacy 
PRO model, we developed the “Planned Resource Optimization Model with 
Experimental Design” (PROM-WED). PROM-WED embeds the legacy PRO 

xvii 



model within a data farming environment. The foundation of PROM-WED’s data 
farming wrapper is the nearly orthogonal Latin hypercube (NOLH). The NOLH 
design of experiments (DOE) builds experimental designs that efficiently and 
effectively explore the solution space (Cioppa & Lucas, 2007). This good space¬ 
filling capability means that uncertainties and fluctuations in input variables along 
with multivariable interactions can be adequately investigated (Sanchez & Wan, 
2015). 

The 33 and 129 design point NOLH designs were used to construct 
PROM-WED’s data farming wrapper. The 33-point NOLH DOE tests each 
variable at 33 levels and grows data for 33 legacy PRO model runs, whereas the 
129-point NOLH DOE tests each variable at 129 levels and grows data for 129 
legacy PRO model runs. PROM-WED’s graphical user interface (GUI) allows 
users to easily input a range of values for each input variable into the NOLH DOE 
worksheet, without need for knowledge or familiarity with data farming or DOE 
techniques (Sanchez, 2011). 

A completed PROM-WED excursion grows a data set for either 33 or 129 
data points. Automatically generated sensitivity analysis provides users with a 
basic risk assessment picture focused on the decision variables using the data 
grown by PROM-WED. Further insights into variable interactions and effects of 
input variables can be easily explored using available data analysis software. 
PROM-WED transforms the legacy PRO model into a resource that N1 can use 
to gain robust insights into the optimal allocation of recruiting resources. 

A scenario of interest to N1 was run and analyzed using PROM-WED. 
Insights gained include: 

1. To optimize the allocation of recruiting resources in fiscal year 2020, it 
is recommended that less funds be allocated to recruiters and more 
funds be allocated to enlistment bonuses and advertisements. 

2. Advertising is the most influential decision variable. Over 80 percent of 
the total cost of recruiting variance is explained by changes in the 
recommended allocation of resources to advertising. 



3. Once relative pay exceeds approximately 1.00, changes in the new 
accession mission have little to no effect on the recommended amount 
of resources allocated to advertising. 
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I. INTRODUCTION 

The mission of the United States Navy is “to maintain, train and equip 
combat-ready Naval forces capable of winning wars, deterring aggression and 
maintaining freedom of the seas” (United States Navy, n.d.-b). Under Title 10 of 
the United States Code, Congress authorizes the Navy to maintain a force- 
strength of over 300,000 active duty personnel to execute this mission 
(Government Accountability Office [GAO], 2016, p. 5). Each year, the Navy 
recruits approximately 35,000 new active duty enlisted Sailors to sustain this 
manning strength (Department of the Navy, 2015, p. 7). The Deputy Chief of 
Naval Operations for Manpower, Personnel, Training and Education (MPT&E/N1) 
is delegated with the responsibility over all Navy manpower readiness matters, to 
include recruiting (United States Navy, n.d.-a). 

Analysts at N1 use mathematical models and simulations to support 
decision makers in the MPT&E domain during the Planning, Programming, 
Budget and Execution (PPBE) process. The PPBE process is the Department of 
Defense’s (DOD) “primary resource management system... for all appropriated 
funding” (Tomasini, n.d.). The DOD’s strategy, force structure, and allocation of 
resources are all delineated within the annual PPBE process (Tomasini, n.d.). 
Each year during the programming phase, N1 submits Program Objective 
Memorandum (POM) inputs, recommending how funds should be allocated 
within the Navy’s MPT&E domain (Defense Acquisition University [DAU], 2013, 
p. 4). 

N1 has historically used the Planned Resource Optimization (PRO) model 
to inform decisions regarding the allocation of recruiting resources and estimate 
total recruiting costs. The PRO model is a deterministic, non-linear optimization 
model that provides users with a recommended set of resources that minimizes 
the cost of recruiting in order to achieve a given recruiting mission. The PRO 
model can also be used to estimate recruiting capacity for a given level of 
resources. 

1 



The PRO model is built in Microsoft Excel using both worksheet functions 
and Visual Basic for Applications (VBA) code. The PRO model’s primary function 
is to provide a broad, estimated budget picture of Navy recruiting resource 
allocation in support of the POM. The PRO model is also used to answer 
questions such as, “what is the least expensive way to meet a recruiting 
mission?” and “how much money do we need to allocate for advertising to meet a 
given accession mission?” (Hogarth, Lucas, & McLemore, 2016, p. 3576) 

A. PROBLEM STATEMENT 

Recruiting a high quality all-volunteer force (AVF) is expensive. The Navy 
requires a growing number of high quality recruits to meet the needs of its 
technologically advanced fleet. In a perfect world, N1 would be given a blank 
check to cover the cost of recruiting a 100 percent high quality force. However, in 
reality, N1 faces a fiscally constrained environment. 

The PRO model is a deterministic, non-linear optimization model that 
provides users with a recommended set of resources that attempts to minimize 
the cost of recruiting (Green & Mavor, 1994). Analysts at N1 use the PRO model 
to optimize resources allocated to advertisements, enlistment bonuses, 
education incentives, and recruiters. Input variables include economic influences 
such as unemployment rate, and policy factors such as target percentage of high 
quality recruits (Navy Recruiting Command, 2007). The PRO model’s primary 
limitations are (1) deviations of input variables are not taken into consideration, 
and (2) extensive experimentation capability is not available. Realistically, the 
input variables to the PRO model fluctuate, are unpredictable, and can interact 
with other variables to influence the recruiting environment and affect the optimal 
allocation of recruiting resources. 

B. THESIS PURPOSE 

The objective of this research is to develop a tool that transforms the PRO 

model into a tool that provides N1 analysts with a robust decision support 

capability for recruiting resourcing decisions. In this research, the author wrapped 
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a design of experiments (DOE) capability around the legacy PRO model. We call 
the enhanced tool the Planned Resource Optimization Model with Experimental 
Design (PROM-WED). 

The data farming wrapper in PROM-WED provides legacy PRO model 
users with the ability to input a range of possible values for input and decision 
variables. The legacy PRO model is run over each scenario that is formulated by 
the DOE tool. Instead of a single, discrete solution found by the legacy PRO 
model, PROM-WED grows data that gives robust insight into cause and effect 
relationships amongst the variables. 

C. RESEARCH QUESTIONS 

In order to best provide N1 analysts with a tool that improves their 
decision support analysis for recruiting budget estimates and resource allocation, 
this research is guided by the following questions: 

1. How can design of experiment techniques better inform decision 
maker’s determination of the optimal and robust combination of 
recruiting resources? 

2. How can efficient design of experiment techniques be incorporated 
around the PRO model for future, on-the-spot risk and sensitivity 
analysis? 

3. Can an enhanced PRO model give decision-makers a robust 
solution for the optimal allocation of recruiting resources? 

D. METHODOLOGY 

PROM-WED’s data farming wrapper uses the Nearly Orthogonal Latin 
Hypercube (NOLH) DOE worksheet tool developed by the Simulation 
Experiments & Efficient Designs (SEED) Center for Data Farming at the Naval 
Postgraduate School (NPS), see https://harvest.nps.edu . A new graphical user 
interface (GUI) allows the user to input a range of values for each input variable 
into the NOLH DOE worksheet. The NOLH DOE worksheet was embedded into 
the PRO model. The user has the option to run an excursion using a 33-point 
design or a 129-point design. PROM-WED generates a robust recommended 
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allocation of recruiting resources. Basic sensitivity analysis provides the user with 
a risk assessment picture, and further analysis can be completed using any data 
analysis software package, such as JMP. Scenarios of interest to N1 are run and 
analyzed. 

E. BENEFITS OF RESEARCH 

The ability to quickly explore scenarios with a deterministic optimization 
model using efficient DOE techniques provides N1 with richer insights into 
combinations of resources that can be utilized to achieve a given active enlisted 
recruiting mission. Instead of a discrete expected value, the implementation of 
efficient DOE techniques provides decision-makers with a “robust [foundation to 
make] decisions or policies” (Sanchez, Sanchez, & Wan, 2014, p. 1). DOE 
methods will also provide improved insight into tradeoff relationships between 
input parameters and the output results (Vieira, Sanchez, Kienitz, & Belderrain, 
2013, p. 264). N1 will also benefit from this study by gaining a tool that provides 
on-the-spot sensitivity analysis using sophisticated DOE techniques. 

F. ORGANIZATION OF THESIS 

This thesis is comprised of five chapters. Chapter I focuses on the 
motivation of the thesis and explains how the research questions are addressed. 
Chapter II discusses the history and composition of the PRO model and 
considers other research that has been done on military recruiting resource 
allocation and the implementation of data farming on simulation models. Chapter 
III addresses the methodology used to build PROM-WED, including a review of 
DOE techniques, and the components of PROM-WED. Chapter IV introduces 
scenarios of interest to N1, and the remainder of the chapter provides an 
analysis of the data generated for these scenarios using PROM-WED. Last, 
Chapter V provides concluding remarks, and recommendations for further work. 
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II. BACKGROUND 


In this chapter, research on military recruiting resource allocation is 
presented, followed by a conceptual overview of the PRO model. 

A. LITERATURE REVIEW 

All military branches face the challenge of determining the best way to 
allocate recruiting resources. PRO is a model that the Navy has historically used 
to help decision makers gain insight to answer this question. 

1. Recruiting Resource Aiiocation Modeis 

Following the United States withdrawal from the Vietnam War in 1973, 
Congress terminated conscription, and the military transitioned to an All- 
Volunteer Force (AVF) (Morey & McCann, 1980, p. 1198). Critics of an AVF were 
concerned that it would result in weakened national security due to low quality 
recruits and insufficient accession numbers. In contrast to a military manned by 
conscripts, an AVF forced each service to expend more effort “to meet the 
various quantity and quality goals” for recruiting new enlistees (Morey & McCann, 
1980, p. 1198). Various modeling efforts were made to gain insight into how to 
best allocate recruiting resources to meet the service’s set recruiting goals. 

In 1978, Chappell and Peel developed static and dynamic optimization 
models to determine the optimal allocation of advertising resources to achieve 
military recruiting goals. The dynamic model they developed introduced 
economic factors such as labor supply and incorporated current and past 
recruiting data to determine an optimal allocation of advertising resources 
(Chappell & Peel, 1978, p. 910). 

In 1980, Morey and McCann developed a model to determine the optimal 
allocation of recruiting resources by inputting econometric data of a given region 
and descriptive data that reflects its demographic population. The model was 
conducive to “perform[ing] sensitivity analyses related to the impacts” of various 
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economic and demographic changes. Their model identified the percentage of 
recruits who graduated from high school as the indicator for recruit quality (Morey 
& McCann, 1980, p. 1204). 

2. Cost-Performance Tradeoff Model 

AVF concerns peaked in 1980, particularly in regards to how the military 
gauged recruit quality (Green & Mavor, 1994, p. 8). The DOD informed Congress 
that the Armed Services Vocational Aptitude Battery (ASVAB), the examination 
used to determine enlistment eligibility, was incorrectly scored between 1976 and 
1980 (Green & Mavor, 1994, p. 2). This error resulted in hundreds of thousands 
of people entering military service who did not meet enlistment standards (Green 
& Mavor, 1994, p. 2). 

Clinical psychologists advised that the recruits who did not meet minimum 
enlistment standards were classified as individuals who “generally need intense 
supervision and guidance, particularly under conditions of serious stress” 
(Laurence & Ramsberger, 1991, p. 8). These attributes are undesirable for 
military service. 

In reaction to this mistake. Congress tasked the DOD “to link enlistment 
standards to job performance” (Green & Mavor, 1994, p. 2). This initiated the 
Joint-Service Job Performance Measurement/Enlistment Standards (JPM) 
Project. The first phase of the JPM project “concentrated on developing a variety 
of measures of job performance so that enlistment standards could be related to 
something close to actual performance on the job” (Green & Mavor, 1994, p. 7). 

Following decades of research, phase one of the JPM project validated 
the ASVAB as “a reasonably valid predictor for performance in entry-level military 
jobs” (Green & Mavor, 1994, p. 10). Fligh school graduation became an indicator 
of likelihood of first term enlistment completion. The military services now faced 
the challenge of determining “how much quality can we afford?” since “high- 
quality personnel cost more to recruit, and the public purse is not bottomless” 
(Green & Mavor, 1994, p. 4,10-11). The goal of the second phase of the JPM 
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project was to address this question. The cost-performance tradeoff (CPT) model 
was their solution (Green & Mavor, 1994, p. 11). 

The CPT model is a tool that decision makers use to estimate the 
“probable effects on performance and/or costs of various scenarios” (Green & 
Mavor, 1994, p. 11). The CPT model is comprised of “four primary components: 
(1) the performance equations, (2) the recruiting cost function, (3) survival rates, 
and (4) training and compensation costs” (McCloy at al.,1992, p. iii). The PRO 
model is based upon the recruiting cost function, which is covered in the next 
section. 

B. PLANNED RESOURCE OPTIMIZATION MODEL 

The PRO model is a non-linear optimization model implemented in 
Microsoft Excel using both worksheet functions and VBA code. It evaluates user 
driven input variables over a recruiting cost function. The result is a 
recommended combination of recruiting resources to meet a given recruiting 
mission. 

1. Components of the PRO Model 

The PRO model uses the recruiting cost function from the CPT model to 
allocate recruiting resources while minimizing the cost of recruiting. A general 
review of the PRO model’s conceptual framework and an overview of the 
workings of the PRO model follow. 

a. Conceptual Framework 

Input variables to the PRO model include decision variables, market 
factors, and policy factors that affect the cost and nature of recruiting. Users can 
change these inputs to test different recruiting scenarios. Figure 1 shows a 
conceptual representation of the PRO model. 
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Figure 1. Conceptual Representation of the PRO Model 


Inputs 

■ Recruiting Mission 

* Percentage of high 
quality recruits 

* Total accession mission 

■ Market Environmental Data 

* Unemployment data 

* Relative pay data 

* Qualified Military 
Available pool 


PRO Model Solving 
Recruiting Cost Function 


Adapted from Navy Recruiting Command (2007, p. 5). 


Outputs 

* Number and Cost of 
Recruiters 

* Advertising Dollars 

* Enlistment Bonus 

* Education Benefits 


A PRO model excursion produces a point solution that tells the user how 
many production recruiters should be in the field along with the allocation of 
funds towards enlistment bonuses, education incentives, and advertisements. An 
example output of a PRO model excursion is shown in Figure 2. 


Figure 2. PRO Model Output 



Resource Run 

2015 

2016 

2017 

2018 

2019 

2020 

2021 


NCO 

35.025 

36.425 

36.800 

35.800 

35.225 

34.650 

34.650 


Capacity 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


Unemployment {%) 

40 

40 

40 

40 

40 

40 

40 

Float 

Total Recruiters 

4 092 

4.632 

4 608 

4.315 

4 106 

3.945 

3.923 


Total Reciuiter Cost (SM) 

S335 138 

S383 495 

$387 265 

$368 083 

$355 535 

$347 480 

$351 645 

Float 

Adii«rtising ($M) 

S86 203 

$105 531 

$105 352 

$97 179 

$91 278 

$87 362 

$87 981 

Fixed 

Enlistment Bonus (SM) 

S40 971 

S36 580 

$41 340 

$40 650 

$42 230 

$42 060 

$42 810 

Float 

Education Incentives ($M) 

SO 000 

SO 000 

$0 000 

$0 000 

$0 000 

$0 000 

$0 000 


LRP (SM) 

S7 440 

S11 220 

$11 280 

$11 380 

$11430 

$11460 

$11 670 


HSDG 

95% 

95% 

95% 

95% 

95% 

95% 

95% 


TSC MIIA 

70% 

70% 

70% 

70% 

70% 

70% 

70% 


Total Cost (SM) 

$469 752 

$536 826 

$545 236 

$517 292 

$500 473 

$488 361 

$494 106 


Program Objective Memorandum (POM) FY17 version of the PRO modei. 


The interpretation of the point solution for fiscal year (FY) 2015 in Figure 2 
is: To meet an accession mission of 35,025 new Navy recruits with a fixed 
eniistment bonus budget of $40,971,000, the Navy should allocate 4,092 Sailors 
to recruiting duty and $86,203,000 to advertising. 
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b. The Recruiting Cost Function 

The recruiting cost function is “the underpinning of the [PRO] model” 
(Navy Recruiting Command, 2007, p. 6). The black box shown previously in 
Figure 1 represents the recruiting cost function. Users cannot alter the recruiting 
cost function to include the elasticities or pre-set data that feed into it. 

The recruiting cost function provides the “minimum cost budget” 
recommended to recruit “a specified number of individuals” while taking into 
consideration the conditions of the “recruiting market” (Green & Mavor, 1994, p. 
126-127). Enlistment supply functions and a constrained minimization problem 
are both critical components of the recruiting cost function (Green & Mavor, 
1994, p. 126-127). 

The variables of interest that build the recruiting cost function include 
(Green & Mavor, 1994, p. 126-127; Katznelson, 2010, p. 4; Navy Recruiting 
Command, 2007, p. 8.): 

H = High quality recruits 
M = Medium quality recruits 
L = Low quality recruits 

C** = Number of contracts signed in a given year, where q = H,M,L 
Cq = Constant (calculated using base year kriowns), where q = H,M,L 

= Miniminn cost to contract given number of recruits, where q = H,M,L 
Aq = Lagrangian multiplier, where q = H,M,L 

a = Elasticity describing the relationship between the paramter and 

= Number of production 7'ecruiters to recruit a given quality, where q = H,M,L 
AD = Advertising dollars spent in the model year (inflation — adjusted to base year) 
B = Average enlistment bonus paid to q = H (model year) 

E = Average education benefits paid to q = H (model year) 

V = Price index to deflate B and E into base year dollai's 

F = Factors affecting the recruiting market (i. e.,unemployment,relative pay) 

T = Testing co5t| 

0 = Fixed costs of recruiting 

The enlistment supply functions, shown in Figure 3, are separated into 

high, medium, and low quality categories of new accessions. These equations 

determine the expected number of recruits in each category that will be 

contracted per year (Green & Mavor, 1994, p. 126). 
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Figure 3. Enlistment Supply Functions 


High Quality Contracts: 
ln(C») = ln(0 + a» 



17/ \17/ 


Medium Quality Contracts: 

InCC'^) = ln(C") + ln(F) 

Low Quality Contracts: 

In(C^) = In(Co) + In(R^) + ap ln(F) 

Adapted from Green & Mavor, 1994, p.126; Navy Recruiting Command (2007, p. 

8 ). 

The “Navy Recruiting Cost Model User Manual” refers to the enlistment 
supply functions as recruiting cost functions (Navy Recruiting Command, 2007, 
p.8). Smith and Flogan refer to these functions as the enlistment supply functions 
in “Modeling Cost and Performance for Military Enlistment, Report of a 
Workshop” (Green & Mavor, 1994, p.126). For the purpose of this research, the 
functions shown in Figure 3 are referred to as enlistment supply functions. 

The objective function shown in Figure 4 determines “the levels of 
recruiting resources and incentives required to recruit the specified mission at 
minimum cost” (Green & Mavor, 1994, p. 127). 

Figure 4. Recruiting Cost Minimization Problem 
Min[p’^{R^ + + R^) + p^°AD + C"(B + E) + r(Cf + C" + C/*) + O] 

-Ah[C" + C"] - - C«] - AJC'' - Ct] 

Source: Green and Mavor (1994, p. 127). 

An explanation of the components that makeup the recruiting “cost 
minimization problem” is shown in Figure 5. 
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Figure 5. Recruiting Cost Minimization Component Roadmap 

First Line: Recruiting Budget Formula 


Cost of Maintaining Cost of Cost of Recruiting Cost of 

Recruiters Advertising Incentives Testing Recruits Fixed Costs 

I -*-1 (—*—I I—^-1 

+ P'"' + + F) + r(C" + + Cj') + O] 

-A„[C« + C«] - - C:^] - AJC^ - C!'] 


Second Line: Constraints to ensure desired recruiting mission is met 

Adapted from Green and Mavor (1994, p. 127). 

The first order conditions of the recruiting minimization problem are then 
“substituted” into the recruiting budget formula (i.e., the first line of the recruiting 
cost minimization problem shown in Figure 5) to “yield the recruiting cost 
function,” shown in Figure 6 (Green & Mavor, 127). 


Figure 6. Recruiting Cost Function 


i+ag+ag 

a 


Minimum Cost Budget = aZ(C") « +p 

+T(C" + C" + O + 0 

where: 




1 
R 


z = 


( u \ —A AD 

(«b) “ («e) “ (f) « 


and a = + or^ 

Adapted from Green and Mavor (1994, p. 127). 
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Elasticities within the recruiting cost function are parameters built into the 
model that “represent the percent change in enlistment contracts for a percent 
change in recruiting resources/incentives or market factors” (McCloy et al., 1992, 
p. 74). The PRO model elasticities were last revised when the SAG Corporation 
and The Lewin Group, Inc. updated the PRO model in September 2011 (Hogan, 
Warner, & Mackin, n.d.). More information about the derivation and specifics of 
the cost performance tradeoff model can be found in the “Job Performance, and 
Cost: A Cost-Performance Tradeoff Model” report and “Modeling Cost and 
Performance for Military Enlistment,” a report of a workshop (McCloy et al., 1992; 
Green & Mavor, 1994). 

c. Model Framework 

The PRO model is made up of four worksheets: (1) User Interface, (2) 
Inputs (will be referred to as the “Data Worksheet” in this research), (3) 
Simulation, and (4) Results. Figure 7 illustrates the conceptual flow and 
relationship between these worksheets. 


Figure 7. Structure of the PRO Model 


/ N 

User Interface Worksheet 






• Input values of interest 



for scenario. 



k_ 

_ ) 



Results Worksheet 

/-;-—^ 

• Displays optimal 
allocation of recruiting 
. resourcesasa point 
solution. 

V 


Adapted from Navy Recruiting Command (2007, p. 12). 


The “User Interface” worksheet allows the user to enter values for each 
input variable, select which decision variables are fixed or to be optimized, and 
select how the model is run over seven FYs. The “Data Worksheet” and the 
“Simulation Worksheet” are components of the “black box” that makeup the 
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recruiting cost function. The “Results Worksheet” provides users with the optimal 
mix of recruiting resources to minimize the cost of recruiting. Each component of 
the PRO model is now explained in more detail. 

2. PRO Model Variables 

The variables of the PRO model are classified into three types: (1) 
decision variables, (2) market factors, and (3) policy factors. Variables are fed 
into the model through PRO’S “User Interface” Microsoft Excel worksheet, as 
shown in Figure 8. 


Figure 8. The PRO Model User Interface 


Step1: Check input 



Model Feeds: 

FY15 FY16 

FY17 

FY18 FY19 

FY20 

FY21 


NCF + College First 

0 000 0.000 

0 000 

0.000 0 000 

0.000 

0 000 


Unemployment Rate 

6.40% 6.10% 

4.90% 

4.80% 4.80% 

4.80% 

4.80% 


Recruiters 

3.913 3.685 

3.685 

3.685 3.685 

3.685 

3.685 


LRP 

7.440 11.220 

11.280 

11.380 11.430 

11.460 

11.670 


Advertising (AC Enl. Only) 

34 826 34 699 

35.679 

36 729 39.886 

38 892 

39.669 


EB 

40 971 36 580 

41.340 

40 650 42230 

42060 

42 810 


NCO (50% BoY DEP) 

' 35,025 ^ 36.425 

^ 36.800 

^ 35.800 ^ 35.225 

^ 34.650 

^ 34.650 


















Step 2: Select switches 










Total Recruiters 

Float 







Fixed 

Advertising 

Float 







Float 

Bonus 

Fixed 








Education 

Float 

























Step 3: Select Best/Worst Cases for Unemployment 

POR +/- Range 

6.10% 


Range (+/-) 

0 50% 




High UE Case 

' 7.40% 

^ 5.90% 

^ 5.80% 

' 5.80% 

^ 5.80% 

^ 5.80% 

POR +/- Range 

Base UE Case 

6.90% 

^ 5.60% 

5.40% 

’■ 5.30% 

" 5.30% 

’■ 6.30% 

’■ 5.30% 

Manual Forecast 

Low UE Case 

' 6.40% 

’■ 5.10% 

’■ 4.90% 

’■ 4 80% 

’■ 4.80% 

’■ 4.80% 

’■ 4.80% 











Manual Forecast 









High UE 

5.00% 

5 00% 

5.00% 

5 00% 

5.00% 

6 00% 

5.00% 


Consensus UE 

4.50% 

4.60% 

4.50% 

4.60% 

4.50% 

4.60% 

4.50% 


Low UE 

4.00% 

4 00% 

4.00% 

4.00% 

4.00% 

4.00% 

4.00% 



















Step 4; Select run type 


















Traditional Run 




capacity Kun 


















POM FY17 version of the PRO model. 


a. Decision Variabies 

The decision variables of the PRO model are the resources that the Navy 

controls. N1 can influence how much is budgeted toward advertising, how many 

production recruiters are in the field, and how funding is allocated for enlistment 
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bonuses and education incentives. It is useful to know the optimal balance 
between these resources, since once money is allocated to a certain resource, 
those funds cannot be used for anything else. For example, funds appropriated for 
recruiters cannot be used to fund advertising (Morey & McCann, 1980, p. 1198). 

(1) Advertising 

Advertising is defined by the United States Government Accountability 
Office (GAO) as “the placement of messages intended to inform or persuade an 
audience through various types of media such as television, radio, digital media, 
direct mail, and others” (GAO, 2016, p. 1). Each military service, including the 
Navy, uses the advertising construct shown in Figure 9 to “raise the public’s 
awareness... and help recruiters generate leads of potential recruits” (GAO, 
2016, p. 1). A lead is someone who shows interest in joining the Navy. Leads can 
be generated a variety of ways, from face-to-face interactions with a recruiter to 
indirect contact through advertisement efforts. 


Figure 9. Phases and Goals of Military Advertisement 


Awareness: 

television commercials, signs, banners 


Engagement: 

websites, digital banner advertisments, social media 




Lead Generation: 

direct mail, recruiting booths, online or print job postings 




Adapted from GAO (2016, p. 7). 


Military advertising efforts are in line with the “consumer journey” construct 
found in the private sector. The goal is “to move a potential recruit through each 
phase and, ultimately, to a decision to enlist” (GAO, 2016, p. 7). The Navy 

14 



















typically allocates about $50 million each FY towards advertising. Table 1 shows 
a breakdown of how much the Navy has annually allotted for advertising over the 
past three FYs in comparison to the other military services (GAO, 2016, p. 41). 


Table 1. Reported Annual Allotments for Military Advertising 



2015 Actuais 

2016 Enacted 

2017 Estimate 

Navy 

$56,100,000 

$49,000,000 

$47,000,000 

Army 

$367,700,000 

$238,100,000 

$292,600,000 

Air Force 

$59,400,000 

$35,900,000 

$60,300,000 

Marine Corps 

$86,300,000 

$81,500,000 

$81,800,000 


These values include active duty and reserve recruiting budgets. Adapted from 

GAO (2016, p. 41). 

(2) Enlistment Bonus 

Enlistment bonuses are used to incentivize high quality applicants to join 
the Navy. Enlistment bonuses are tied to specific Navy occupational specialties. 
Special warfare (Navy sea, air, and land (SEAL) operators) and nuclear field (NF) 
specialties are examples of Navy occupational specialties that require high 
ASVAB scores, and regularly offer an enlistment bonus to recruits. Figure 10 
shows the individual enlistment bonus offered to each recruit who enlisted as a 
special operator or in the nuclear field. Enlistment bonuses fluctuate for many 
reasons, to include: the time of year the recruit ships to boot camp, and under or 
over manning strength of the Navy occupational specialty (Navy Recruiting 
Command, n.d.). 
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Figure 10. Enlistment Bonus Offered to NF and Special Operator 

Recruits 



Data gathered from Navy Recruiting Command enlistment bonus messages from 
October 2002 to February 2016. Adapted from Navy Recruiting Command (n.d.). 


(3) Production Recruiters 

Production recruiters are Navy Sailors assigned to recruiting duty with the 
9585 Navy Enlisted Classification (NEC) code. For model simplicity, the PRO 
model divides production recruiters into three quality categories: low, medium, and 
high. The model designates high quality recruiters to recruit only high quality 
applicants, and so on. Navy Recruiting Command (NRC) does not separate 
recruiters into these tiered categories. This simplifying assumption helps represent 
that “high-quality personnel cost more to recruit” (Green & Mavor, 1994, p. 10-11). 

The number of production recruiters in the field directly affects the total 
cost of recruiting. The PRO model takes recruiter’s base pay and individual 
support costs into consideration when calculating the cost of recruiting. 

(4) Education Incentive 

Following the enactment of the Post 9/11 Gl Bill in June of 2008, the Navy 
has not allocated funds towards the legacy Navy college fund (NCF) for new 
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recruits (Dortch, 2014, p. 1; Palmer, personal communication, June 2016). The 
Post 9/11 Gl Bill provides service members who have served on active duty 
following September 10, 2001, with education benefits that “can cover all in-state 
tuition and fees at public degree granting schools” along with support programs 
for out-of-state and private institutions (Department of Veterans Affairs, 2012). 

b. Market Factors of the PRO Model 

Market factors of the PRO model are economically and demographically 
driven variables that are uncontrollable and subject to unexpected change. 

(1) Unemployment Rate 

Unemployment rates are shown to be “positively and significantly related 
to high-quality enlistment contracts” (Asch et al., 2010, p. 21). As shown in Figure 
8, this trend indicates that higher unemployment rates lead to more high quality 
enlistment contracts (Bicksler & Nolan, 2009, p. 5). Figure 11 shows data 
aggregated for all military services. 


Figure 11. Unemployment and High-quality Enlistments 



85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 

Year 


Source; U5. Department of Defense and U.S. Bureau of Labor Statistics 
Note: High-quality enlistments are high school graduates who score 
at or above average on the Armed Forces Qualification TesL 

Source: Bicksler and Nolan (2009, p. 5). 
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(2) Relative Pay 

Like unemployment rates, military recruiting for high quality applicants 
responds “to the level of military pay relative to civilian sector wage opportunities” 
(Warner, 2012, p. 71). The PRO model captures this market driver through the 
relative pay ratio. Since the Navy requires high quality Sailors who are technically 
competent, “pay comparability... is an issue for certain hard-to-fill occupations 
and skills that command high salaries in the civilian sector, particularly in high 
technology fields” (Bicksler & Nolan, 2009, p. 34). Table 2 provides 
interpretations of relative pay ratios. 


Table 2. Interpretations of Relative Pay 


Relative Pay 

Interpretation 

0.5 

Military pay is 50 cents to every dollar of civilian sector pay. 

1.0 

Military pay is equal to civilian sector pay. 

2.0 

Civilian sector pay is 50 cents to every dollar of military pay. 


(3) Qualified Military Available 

Qualified military available (QMA) is an estimate of the “17- to 24-year-old 
youth population in the United States who would qualify without needing a waiver 
and be available to enlist in the active component military” (Qffice of the 
Undersecretary of Defense, 2016, p. 2). An independent firm. Woods and Poole 
Economics, provides the Navy with QMA data (www.woodsandpoole.com). 

Some common disqualifiers for applicants joining the military include; illicit 
drug use, overweight/obesity, use of prescribed psychotropic drugs, and failure to 
complete high school. Figure 12 is a hypothetical model which depicts the QMA 
pool for the recruiting efforts of the four military services. The resulting pool of 
QMA is just a small portion of the overall military-aged population within the 
United States. 


18 




Figure 12. Hypothetical Breakdown for Estimated QMA Pool in 2030 


In 2030 there will be approximately 30 million military 
aged (18 - 24) people in the United States 



Data approximations are adapted from MarketingCharts (2016); Child Trends Data 
Bank (2014); Child Trends Data Bank (2016); National Center for Education 
Statistics (2016). Propensity to serve metric is omitted due to distribution 
restrictions. 


Other factors that are not considered include the percentage of young 
adults who are currently enrolled in college, those who are permanently 
employed, or those who may have dependents. 

c. Policy Factors of the PRO Model 

Policy factors are variables that can be adjusted, but are done through a 
combination of policy, service culture, and budget changes. 

(1) New Contract Objective 

The new contract objective (NCO) is the Navy’s enlisted accession 
mission for a given FY. NRC’s NCO goal each FY is dependent on the Navy’s 
projected end-strength. The equation for end-strength is shown in Figure 13. 
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Figure 13. Equation for Navy Enlisted Strength Planning 


F. Fhxtoh .H 


Begin 

Strength 

- 

Losses 

+ 

Gains 

• Prior-year end 


• Natural losses 

• Enlisted 

strength 


• Force shaping 

accessions 


losses 


F--- mandate ---H 

End 

Strength 

• End-of-year 
strength 

• Driven by NDAA/ 
fiscal controls 


Adapted from Dave Clark’s 2015 presentation at the Naval Postgraduate School 
(personal communication, (September 10, 2015). 


Force planners at N1 forecast the number of Sailors who will leave the 
Navy each year; referred to as manning losses. NRC’s NCO goal ensures that 
the Navy’s force strength meets the congressionally mandated end-strength for 
each FY (Clark, personal communication, 2015). Table 3 demonstrates the 
different phases of Navy manning within a FY. The NCO mission is dynamic and 
often fluctuates throughout the FY in response to actual manning losses. 


Table 3. Navy Manning Terminology 


Terminology 

Description 

Begin Strength 

Current onboard as of October 1 of current FY 

Force Strength 

Current onboard anytime between October 2 
and September 29 of current FY. 

End Strength 

Current onboard as of September 30 of current 
FY 


The first day of the FY is October 1 and the last day of the FY is September 30. 


(2) Loan Repayment Program 

The loan repayment program (LRP) is an incentive that the Navy uses to 
attract high quality applicants with student loan debt to enlist in specific 
occupational specialties. Assuming enlisted service members with less than one 
year of service pursued higher education prior to joining the military. Figure 14 
indicates that approximately 60 percent of recruits across all military branches 
have attended at least some college before joining the military. Figure 14 
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includes credits towards an undergraduate degree, a completed undergraduate 
degree, and postgraduate education. 


Figure 14. Education Levels of Enlisted Personnel 



H High-school I Some college H 4YR degree+ 


Assuming that service members who have between 0-1 year of service eniisted 
into the service with the education level shown. Source: Grefer, Gregory, and 
Rebhan (2011, p. 10). 

Through the LRP, the Navy “pay[s] federally guaranteed student loans (up 
to $65,000) through three annual payments during a Sailor’s first three years of 
service” (Navy Recruiting Command, 2017). Student loan debt “is the only form 
of consumer debt that has grown since the peak of consumer debt in 2008,” as 
shown in Figure 15 (Lee, 2013, p. 5). 
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Figure 15. Rise of Student Debt 
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Source: Lee (2013, p. 5). 


Consequently, the military’s loan repayment program may be an attractive 
option for a high quality recruit with student loan debt. 

(3) High School Diploma 

Studies indicate that a high school diploma is “a valuable predictor of 
military attrition” (Buddin, 1984, p. 2). Recruits who do not have a high school 
diploma are more likely to not finish their initial obligated military service. In 
response to this “well-known result,” the DOD has a benchmark that at least 90 
percent of new accessions must join the military with a high school diploma 
(Navy Recruiting Command, 2007, p. 7; Buddin, 1984, p. 1). A general education 
development (GED) certificate is not considered a high school diploma (Buddin, 
1984, p. 1). 

(4) Recruit Quality 

Recruit quality is determined by an applicant’s Armed Forces Qualification 
Test (AFQT) score. The AFQT score is derived from the ASVAB’s “Arithmetic 
Reasoning (AR), Mathematics Knowledge (MK), Paragraph Comprehension (PC), 
and Word Knowledge (WK)” subsections (Defense Management Data Center, 
n.d.). 
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High school graduates who earn above a 50 AFQT are classified as “high 
quality” applicants (Navy Recruiting Command, 2007, p. 6). The term high quality 
is also referred to as Test Score Category (TSC) l-IIIA. This group is represented 
by the “A” block in Figure 16. 

Figure 16. Navy Recruit Quality Determination 


AFQT 

TSC 

High School 
Diploma 

No High School 
Diploma 

99-93 

1 

A 

B 

93-65 

II 

65-50 

IMA 

50-31 

IIIB 

Cu 

D 

31-0 

IVa 

Cl 

IVb 

IVc 

V 

1 DOD Ineligible | 


Adapted from Navy Recruiting Command (2007, p. 7). 

The Navy aims to recruit applicants who meet the group “A” requirement 
because they qualify for most Navy occupational specialties, have the lowest first 
term attrition rate, historically encounter fewer disciplinary problems, and are likely 
to have the best career performance. However, this category of applicants tends to 
be the most expensive to recruit (Navy Recruiting Command, 2007, p. 7). 

High quality applicants typically have multiple opportunities, such as 
college or a well-paying job. Therefore, the Navy must invest more money in 
advertisements targeting group “A,” increase enlistment bonuses to incentivize 
group “A,” and increase recruiting manpower to recruit group “A” applicants. 
Each of these contribute to the high cost of recruiting high quality applicants. 

Descriptions and characteristics of all categories represented in Figure 16 
are explained in Table 4. 
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Table 4. Recruit Quality Category Description 


Block/Category 

Description 

A 

(1) Qualify for the most amount of programs 

(2) Have the lowest first term attrition 

(3) Encounter fewer disciplinary problems 

(4) Likely to have the best career performance 

B 

(1) Highest first term attrition rate 

(2) Qualify for many programs 

Cu 

(1) Attrition lower than “B,” but higher than “A” 

(2) Applicants do not qualify for many programs. 

Cl 

Navy does not recruit from this group 

D 

Navy does not recruit from this group 


Adapted from Navy Recruiting Command (2007, p. 7). 


d. PRO Model: Run Options 

PRO model excursions can be run two different ways; (1) traditional run, 
or (2) capacity run. 

(1) Traditional Run 

The traditional run option of the PRO model performs an optimization that 
minimizes the cost of recruiting by determining the optimal allocation of resource 
spending to advertisements, enlistment bonuses, education incentives, and 
recruiters. The traditional run can be evaluated as either an unconstrained or a 
constrained problem. 

An unconstrained traditional run does not bound any of the decision 
variables. The result is “an unconstrained, minimum cost solution” (Navy 
Recruiting Command, 2007, p. 22). Unconstrained traditional runs may produce 
results that are mathematically feasible, but are infeasible in practice. For example, 
while it would be unrealistic for NRC to have more than 4,000 recruiters in the field, 
the PRO model may determine 4,520 recruiters to be the optimal solution. 

Figure 17 shows the results of an unconstrained traditional run of the PRO 
model. The highlighted rows are the results. In FY 2015, with a 5 percent 
unemployment rate, and a recruiting mission of 34,000, the optimal allocation of 


24 





recruiting resources to minimize the cost of recruiting was: assign 3,137 Sailors 
to recruiting duty, allocate $50,960,000 to advertising, and $67,267,000 to 
enlistment bonuses. 

Figure 17. Results of an Unconstrained PRO Model Run 



Resource Run 

2015 

2016 

2017 

2018 

2019 

2020 

2021 


NCO 

34.000 

34.000 

34.000 

34.000 

34.000 

34.000 

34.325 


Capacity 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


Unemployment (%) 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

Float 

Total Recruiters 

3.137 

3.132 

3.125 

3.117 

3.110 

3.101 

3.141 


Total Recruiter Cost {$M) 

$256,944 

$269,364 

$262 626 

$265 932 

$269281 

$273,186 

$281,505 

Float 

Advertising ($M) 

$50 960 

$51,329 

$51,798 

$52,272 

$52,750 

$53,293 

$55,170 

Float 

Enlistment Bonus ($M) 

$67,267 

$67,754 

$68,374 

$68 999 

$69,630 

$70,346 

$72825 

Float 

Education Incentives (SM) 

$0,000 

$0 000 

$0 000 

$0 000 

$0,000 

$0 000 

$0,000 


LRP ($M) 

$7,440 

$11,220 

$11,280 

$11,380 

$11,430 

$11,460 

$11,670 


HSDG 

95% 

95% 

95% 

95% 

95% 

95% 

95% 


TSC l-IIIA 

70% 

70% 

70% 

70% 

70% 

70% 

70% 


Total Cost ($M) 

$382 611 

$389 667 

$394 078 

$398 683 

$403,091 

$408,286 

$421,170 


POM FY17 version of the PRO model. 


In contrast, a constrained traditional run fixes at least one of the four 
decision variables. The decision variables that are fixed remain constant. The 
remaining unconstrained decision variables are optimized (Navy Recruiting 
Command, 2007, p. 22). 

Figure 18 demonstrates the results of a constrained traditional PRO model 
run where advertising and enlistment bonus were fixed and the total number of 
recruiters was optimized. 


Figure 18. Results of a Constrained Traditional PRO Model Run 



Resource Run 

2015 

2016 

2017 

2018 

2019 

2020 

2021 


NCO 

34,000 

34,000 

34,000 

34,000 

34,000 

34,000 

34,325 


Capacity 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


Unemployment (%) 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

Float 

Total Recruiters 

3,037 

3.042 

3,047 

3,052 

3.058 

3,063 

3,151 


Total Reciuter Cost ($M) 

$248,724 

$251,880 

$256,097 

$260,386 

$264,747 

$269,825 

$282408 

Fixed 

Advertising ($M) 

S60.000 

$60,000 

$60,000 

$60,000 

$60,000 

$60,000 

$60,000 

Fixed 

Enlistment Bonus ($M) 

$67,500 

$67,500 

$67,500 

$67,500 

$67,500 

$67,500 

$67,500 

Float 

Education Incentives ($M) 

$0,000 

$0 000 

$0 000 

$0,000 

$0,000 

$0,000 

$0,000 


LRP ($M) 

$7,440 

$11,220 

$11,280 

$11,380 

$11,430 

$11,460 

$11,670 


HSDG 

95% 

95% 

95% 

95% 

95% 

95% 

95% 


TSC HIIA 

70% 

70% 

70% 

70% 

70% 

70% 

70% 


Total Cost ($M) 

$383,664 

$390,600 

$394,877 

$399,266 

$403677 

$408,785 

$421,578 


POM FY17 version of the PRO model. 
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The highlighted rows are the results. In FY 2015, with a 5 percent 
unemployment rate, a recruiting mission of 34,000, advertising fixed at 
$60,000,000, and enlistment bonuses fixed at $67,500,000, the Navy should 
assign 3,037 Sailors to recruiting duty. 

(2) Capacity Run 

The capacity run estimates the number of recruits the Navy can expect to 
recruit based on a predetermined allocation of recruiting resources. Figure 19 
exhibits the results of a capacity run. 


Figure 19. Results of a Capacity PRO Model Run 


Capacity Run 

2015 

2016 

2017 

2018 

2019 

2020 

2021 

NCO 

34,000 

34.000 

34,000 

34,000 

34,000 

34.000 

34.325 

Capacity 

33.426 

33.405 

33,385 

33,364 

33.344 

33,324 

33,303 

Unemployment (%) 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

Total Recruiters 

2.900 

2.900 

2.900 

2.900 

2.900 

2.900 

2,900 

Total Recruiter Cost ($M) 

$237,520 

$240 119 

$243,725 

$247,381 

$251 093 

$255,477 

$259 917 

Advertising ($M) 

$60,000 

$60 000 

$60,000 

$60 000 

$60,000 

$60,000 

$60,000 

Enlistment Bonus ($M) 

$67,500 

$67 500 

$67,500 

$67 500 

$67,500 

$67,500 

$67,500 

Education Incentives ($M) 

$0 000 

$0,000 

$0 000 

$0,000 

$0,000 

$0,000 

$0,000 

LRP ($M) 

$7,440 

$7,440 

$11,220 

$11,280 

$11,380 

$11,430 

$11,460 

HSDG 

95% 

95% 

95% 

95% 

95% 

95% 

95% 

TSC l-IIIA 

70% 

70% 

70% 

70% 

70% 

70% 

70% 

Total Cost ($M) 

$372,460 

$375,059 

$382 445 

$386 161 

$389 974 

$394,407 

$398 877 


POM FY17 version of the PRO model. 


The results indicate that in FY 2015 the Navy can expect 33,426 recruits 
with 2,900 recruiters in the field, $60,000,000 allocated to advertising, and 
$67,500,000 allocated to enlistment bonuses. 

3. Updates to the PRO Model 

In 2011 the SAG Corporation and The Lewin Group, Inc. updated the PRO 
model based on specific shortcomings of the model identified by the Navy (Hogan 
et al., n.d., p. 1). The updated model is referred to as the Recruiting Program 
Resource Optimization (E-PRO) model. The E-PRO model added “stochastic 
forecasting capability” and updated the econometric elasticities within the recruiting 
cost function (Hogan et al., n.d., p. 3). 
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Despite these updates, analysts at N1 still use the PRO model since it is 
“simpler in construct [compared to E-PRO]... and delivers very good results” 
(Palmer, personal communication, April 7, 2016). As mentioned earlier, the current 
version of the PRO model uses “the pooled baseline elasticities updated from 
the... [2011] E-Pro effort” (Palmer, personal communication, April 12, 2016). 

4. Limitations of the PRO Modei 

The existing PRO model does not have the capability to efficiently test 
uncertainties in variable values, or the effects of variable interactions. Without this 
capability, PRO model users must use either manual trial and error techniques to 
test different scenarios individually, or build macros in Excel to test the fluctuation 
of a single variable. For example, a macro was written to test three levels of 
unemployment rate, as shown in Table 5. 


Table 5. Pooled Unemployment Rates 



FY 15 

FY16 

FY17 

FY18 

FY19 

FY 20 

FY21 

High UE 

5.00% 

5.00% 

5.00% 

5.00% 

5.00% 

5.00% 

5.00% 

Base UE 

4.50% 

4.50% 

4.50% 

4.50% 

4.50% 

4.50% 

4.50% 

Low UE 

4.00% 

4.00% 

4.00% 

4.00% 

4.00% 

4.00% 

4.00% 


Adapted from POM FY17 version of the PRO model. 


Without options for multivariable sensitivity analysis or efficient 
experimentation, it is difficult to understand how variable interactions or 
fluctuations in controllable and uncontrollable factors affect the model’s output. 
This may be an area of concern when the output is used to help inform decisions 
involving hundreds of millions of dollars. 
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III. METHODOLOGY AND IMPLEMENTATION 


This chapter covers three main topics that span the motivation, design, 
and implementation of PROM-WED. First, an overview of design of experiments 
techniques is presented. Next, the field of data farming is introduced to include 
examples of past research studies that have utilized data farming. Finally, these 
two concepts are integrated as the design and construction of PROM-WED is 
explained. 

A. DESIGN OF EXPERIMENTS 

The objective of an experiment across any discipline of study is “to 
investigate characteristics of a system” (Park, 2007, p. 309). There are no limits 
to what this system can be, from the test and evaluation of a new military 
warship, to sensitivity analysis on a political science poll. Every system has 
inputs and outputs. Inputs are either controllable or uncontrollable. Controllable 
factors are input variables to the system that are known and can be set, such as 
the number of Navy destroyers that enter a theater of operations in a combat 
simulation, to the number of production recruiters Navy Recruiting Command has 
in the continental United States. Uncontrollable factors are input variables to the 
system that are uncertain, such as the unemployment rate in 2021, or the 
probability of kill for an adversary’s new weapon system. A general model of a 
system is shown in Figure 20. 


Figure 20. General Model of a System. 


Inputs ^ 


Controllable 


Outputs 



Uncontrollable 


Adapted from Penn State (n.d.). 
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Early development in DOE methodology occurred predominately in the 
physical sciences, specifically in agriculture (Penn State, n.d.)- The classical 
methods and foundations of DOE can also be applied to the testing and analysis 
of simulation models (Sanchez, 2006, p. 69). Control, replication, and 
randomization are considered to be “fundamental concepts” of DOE (Sanchez, 
2006, p. 69). Working definitions of these concepts in the context of DOE are 
shown in Table 6. 


Table 6. Fundamental Concepts of DOE. 


Fundamental 

Concept 

Working Definition 

Control 

“The experiment is conducted in a systematic manner after 
explicitly considering potential sources of error, rather than by 
using a trial-and-error* approach. ” 

Replication 

“A way to gain enough data to achieve narrow confidence 
intervals and powerful hypothesis tests.” 

Randomization 

“Provides a probabilistic guard against the possibility of unknown, 
hidden sources of bias surfacing to create problems with your 
data.” 


Adapted from Sanchez (2006, 69). 


To adequately test a system, whether the system is a simulation model or 
a physical science experiment, trial-and-error should be avoided. Trial-and-error 
is inefficient, difficult to replicate, and lacks control. DOE techniques combat 
these limitations through systematically testing a model with control, replication, 
and randomization. Systematic approaches are also conducive for automation, 
which alleviates manual work, and increases the efficiency and capability of the 
system being explored. The automation of DOE techniques has created the field 
of data farming, which is further explained in the next section. 

There are many different DOE methods and techniques available, such as 
the full and fractional factorials, central composite designs, and nearly orthogonal 
Latin hypercubes (NOLHs). The full factorial and NOLH methods are explained in 
further detail. More information regarding DOE basic concepts, methods and 
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their application to simulation modeling can be found in Sanchez and Wan’s 
report, “Work Smarter, Not Harder: A tutorial on designing and conducting 
simulation experiments” (Sanchez & Wan, 2015). 

1. Full Factorial DOE Method 

The full factorial approach tests every possible combination of input 
factors given fixed levels. The classic game of “capture the flag” is used to 
explain the full factorial method. The objective of the game is for a member of 
one team to capture a flag that is kept on the other side of the field, and return it 
to their side of the field. If caught by a member of the opposing team on the 
opposition’s side of the field, the player fails the mission, and is temporarily 
placed in “jail.” Figure 21 shows a simple representation of the “capture the flag” 
game, where the gray team on the left is trying to capture the gray flag on the 
opposition’s side, and vice versa. 


Figure 21. Capture the Flag Game 



The circles represent the players of each team. Adapted from MultiCulturalGames 
(n.d.). 

Two attributes that may affect the success of a “capture the flag” player 
are speed and stealth. Figure 22 illustrates the testing of various degrees of 
speed and stealth for a “capture the flag” player. The sparse grid on the left tests 
the system only at its extreme values, where either minimum speed or minimum 
stealth results in a failure, but maximum speed and maximum stealth results in a 
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success. The grid on the right demonstrates a dense full factorial test where 
many possible levels of stealth are tested against many possible levels of speed. 
In this hypothetical example, success can be met at something other than a 
combination of full speed and full stealth (Sanchez & Wan, 2015, p. 1801). 


Figure 22. A Sparse versus a Dense Full Factorial DOE for 

Capture the Flag 



ca 

O 

C/D 


■ ▼▼▼▼▼ ooooo 

▼▼TVOOOOO 

■ T TVTVOOOOO 
■TTTVTOOOOO 
▼TTTOOOOO 
▼▼▼▼OOOOO 
■TTVTTOOOOO 
■TTTVTTOOOO 
iBTTTTTTTOO 



Speed 


The block shapes indicate failure, whereas the circle indicates success. The 
triangle represents a result somewhere in between failure and success. Source: 
Sanchez and Wan (2015, p. 1801). 


The dense full factorial grid in Figure 22 illustrates two key advantages 
that DOE techniques can offer: (1) space-filling capability, and (2) robust insight 
and understanding of the solution space. 

Space-filling refers to a DOE’s capability of testing the simulation over a 
broad spectrum of input combinations (Sanchez & Wan, 2015). Figure 22 
demonstrates that testing only the maximum and minimum values does not have 
good space-filling capability, whereas using the multi-level full factorial DOE 
exemplifies high space-filling capability. The ability to test a factor at different 
levels increases the potential insight gained from the solution space (Sanchez & 
Wan, 2015). As demonstrated in Figure 22, the space-filling DOE provides 
insight to capture the flag players that the right combination of stealth and speed 
resources can achieve the target solution using less resources. Full factorial 
DOES are orthogonal, which means that there are no confounding effects. 
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Certain combinations of input variables, such as speed and stealth, may 
influence the effect of each other. This is referred to as a variable interaction. 
Variable interactions identify “whether the levels of some factors influence the 
effects that other factors have” on the solution (Sanchez & Wan, 2015, p. 1796). 
Without proper care in designing experiments, interactions can be impossible to 
estimate. 

Time and computing capability can quickly become limiting factors when 
performing DOE tests on complex simulation models. Testing a complex model 
using a full factorial of all possible combinations of variables is inefficient and 
often inconceivable. For example. Table 7 demonstrates how a DOE that 
examines a model with only 20 factors can quickly become infeasible as the 
number of levels increases. 


Table 7. Number of Experiments Required to Test a Model with 20 

Factors Using Full Factorial Designs 


Number of levels each factor is 
studied at 

Equation 

Number of Experiments Required 

2 

(i.e., only a min and max value) 

o 

CN 

C\J 

1,048,576 

4 

(i.e., min, max and 2 values in 
between) 

o 

CN 

1,099,510,000,000 

6 

(i.e., min, max and 4 values in 
between) 

o 

CN 

CD 

365,616,000,000,000 


Adapted from Sanchez (2006, p. 76). 

Increasing the number of experiments becomes costly since more 
experimental runs require higher computing capability, and increased work 
hours. 
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2. Nearly Orthogonal Latin Hypercubes 

Cioppa and Lucas (2007) developed the nearly orthogonal Latin 
hypercubes (NOLH) which are efficient and effective alternatives to the full 
factorial DOE. “Latin hypercube designs have proven useful for exploring 
complex, high-dimensional computational models, but can be plagued with 
unacceptable correlations among input variables” (Hernandez, Lucas, & Carlyle, 
2012, p. 1). Cioppa and Lucas’ work addresses this problem by “inducing small 
correlations between the columns in the design matrix” (2007, p. 45). The result 
is the nearly orthogonal Latin hypercube. These NOLH DOEs provide analysts 
with many advantages, including the ability: 

to determine the driving factors, detect interactions between input 
variables, identify points of diminishing or increasing rates of return, 
and find thresholds or change points in localized areas... [and] fit 
many diverse metamodels to multiple outputs with a single set of 
runs. (MacCalman, Vieira, & Lucas, 2016, p. 1) 

Figure 23 shows a comparison of space-filling capabilities between two full 
factorial designs (A and B), versus two NOLH designs (C and D). The four 
designs are respectively a 2"^ and a 4"^ full factorial designs, and a 17-point and 
257-point NOLH DOEs. 


Figure 23. 
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Source: Sanchez and Wan (2015, p. 1802). 


Table 8 provides a numerical representation of the four DOE designs 
shown in Figure 23. For one extra design point (i.e., 16 to 17, or 256 to 257), we 

get much greater space filling with the NOLH DOE. 
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Table 8. Factorial Designs versus NOLH Designs 


Pairwise Piot 
Matrix 

Design 

Factors 

Levels 

Design 

Points 

A 

2^^ Factorial 

4 

2 

16 

B 

4"^ Factorial 

4 

4 

256 

C 

NOLH 

7 

17 

17 

D 

NOLH 

29 

257 

257 


Adapted from Sanchez and Wan (2015, p. 1802). 


As demonstrated by Figure 23 and Table 8, the NOLH designs minimize 
computational effort while improving space-filling capability, allowing for more 
factors to be tested within the same experimental design (Sanchez & Wan, 2015, 
p. 1803). At the cost of one additional design point, we are able to analyze 7 or 
29 factors at 17 and 257 levels, respectively, in comparison to a factorial design 
with 4 factors at either 2 or 4 levels. Reference Cioppa and Lucas’ paper 
“Efficient Nearly Orthogonal and Space-filling Latin Hypercubes” for more 
information about the NOLH DOE method (Cioppa & Lucas, 2007). 

From the initial research done by Cioppa and Lucas, other families of 
NOLH designs have been developed to enhance and make the NOLH designs 
adaptable to further applications in simulation analysis. To expand the NOLH 
designs capability a mixed integer program (MIP) algorithm was developed “that 
generates Latin hypercubes with little or no correlation among their columns for 
most any determinate run-variable combination” (Hernandez et al., 2012, p. 1). 
This MIP algorithm is also adaptable and accommodating to run modifications. 
(Hernandez et al., 2012, p. 1). A second-order NOLH design has also been 
developed that facilitates “exploratory analysis of stochastic simulation models in 
which there is considerable a priori uncertainty about the forms of the responses” 
(MacCalman et al., 2016, p. 1). Lastly, Sanchez created a Microsoft Excel 
spreadsheet that uses Cioppa and Lucas’ NOLH DOE algorithm to provide users 
with the ability to generate designs ranging from simple small orthogonal Latin 
hypercubes to complex NOLH designs that handle up to 29 factors at 257 levels 
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each (Sanchez, 2011). These designs, along with other DOE methods, are 
available in Microsoft Excel format at https://harvest.nps.edu. 

B. DATA FARMING 


Work smarter, not harder... 

—Professor Susan Sanchez (2006) 


The use of robust design of experiment techniques has spawned a field of 
data analytics for simulation models, referred to as data farming. In comparison 
to traditional methods such as data mining, where one “seek[s] to uncover 
valuable nuggets of information buried within massive amounts of data,” data 
farming grows data by controlling the interactions of the variables through 
efficient DOE techniques (Sanchez, 2014, p. 800). Retrospective data collection 
can find correlations, but prospective DOE is required to establish causality. 

Data farming is an iterative process that allows analysts to gain robust 
insight into the ‘“big picture’ solution landscape” (Horne & Meyer, 2010, p. 1). Six 
foundational components of data farming are shown in Figure 24. 


Figure 24. The Six Realms of Data Farming 
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Source: Horne and Meyer (2010, p. 2). 


36 









Steve Upton of the SEED center at NPS has built multiple data farming 
wrappers to facilitate efficient DOE testing around simulation models spanning 
diverse computing environments and subject areas. The data farming wrappers 
that he builds are computer programs that wrap a DOE algorithm around a pre¬ 
existing model. The following is a sample of research that utilizes Upton’s data 
farming wrappers: 

1. Erin Borozny tested the Navy’s Officer Strategic Analysis Model 
(OSAM) using data farming. OSAM is a manpower model that 
projects officer end strength and force structure based on 
“personnel plans and force-shaping policy” (Borozny, 2015, p. v). 
Her research provides insight into effective ways the Navy can 
better manage its officer inventory in order to meet authorized end 
strength at the end of each FY (Borozony, 2015). 

2. Christian Seymour applied data farming to the Synthetic Theater 
Operations Research Model (STORM). The Department of Defense 
uses STORM as its “primary campaign analysis tool” that considers 
“force structures, operational concepts, and military capabilities” 
(Seymour, 2014, p. v). His study shows that data farming 
“capitalize[s] on STORM’s full potential” and provides policy makers 
with robust insights in an efficient and effective manner (Seymour, 

2014, p. v). 

3. Jeffery Parker’s research on the Marine Corps’ future amphibious 
capability used data farming around a model that simulated 
amphibious assaults. His research provides informative decision 
support for United States Navy procurement “by evaluating the 
[Marine Expeditionary Unit’s] MEU’s expeditionary amphibious 
assault capability and the use of ship-to-shore connectors” (Parker, 

2015, p. v). 

These are only three examples of numerous studies that have utilized a 
data farming wrapper around a simulation model. They demonstrate how 
adaptable, capable, and valuable data farming an existing model can be. For 
more information about studies that have used data farming in defense 
applications, visit https://harvest.nps.edu . 
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c. 


PROM-WED 


PROM-WED was developed to provide analysts with a tool that evaluates 
the PRO model over scenarios constructed using the NOLH DOE algorithm. 
PROM-WED also provides analysts with decision support capabilities that 
capitalize on its ability to grow data, and perform sensitivity and risk analysis to 
better inform decision makers on a robust solution to the optimal allocation of 
recruiting resources. PROM-WED excursions can be run to model the effects of 
varying degrees of policy changes and a range of economic and demographic 
conditions that affect the total cost of recruiting. One PROM-WED excursion 
provides decision support analysis to cover the effects of all of these factors and 
their interactions with one another. 

To achieve these objectives, PROM-WED is divided into three main 
components: (1) the NOLH DOE data farming wrapper, (2) the GUI, and (3) 
decision support analysis. For the purpose of this research, focus is placed on 
the traditional run option. Refer to Chapter V regarding the capacity run option. 

Since the PRO model is built in Microsoft Excel, PROM-WED is also built 
in Microsoft Excel, specifically Microsoft Excel 2013 Version 15.0.4849.1003 
(Microsoft Excel, 2013). Given the restrictions and limitations of software allowed 
on government computers, maintaining PROM-WED in the Microsoft Excel 
environment allows accessibility of use to any government computer without 
requiring any additional software. 

1. Data Farming Wrapper 

The NOLH DOE algorithm is the foundation of PROM-WED’s data farming 
wrapper. The NOLH was chosen for its space-filling capability and ease of use in 
a Microsoft Excel VBA modeling environment. The SEED Center at NPS has 
made the NOLH DOE algorithm available in a Microsoft Excel worksheet at 
https://harvest.nps.edu. 

PROM-WED’s data farming wrapper uses both the 33-point and 129-point 

NOLH design worksheets. The 33-point design tests up to 11 variables at 33 
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levels, whereas the 129-point design tests up to 22 variables over 129 levels. 
The 129-point design has better space-filling properties, but takes more time to 
run. Figure 25 shows a pairwise plot comparison of the space-filling ability of 
these two designs. The user is able to choose which NOLH design they want to 
run excursions over using the GUI that is further explained in the next section. 


Figure 25. Pairwise Plots for the 33 and 129 Point NOLH Designs 



Left: 33-point NOLH DOE. Right: 129-point NOLH DOE. 


Table 9 shows an example PROM-WED test case scenario. 


Table 9. Example PROM-WED Scenario 


Variable Type 

Variable Name 

Value Low 

Value High 

Decision Variable 

Production Recruiters 

2,500 recruiters 

3,500 recruiters 

Market Factor 

Unemployment Rate (UE) 

4.0% 

8.0% 

Market Factor 

Relative Pay 

0.8 

1.2 

Policy Factor 

Recruiting Mission (NCO) 

30,000 recruits 

40,000 recruits 


Figure 26 shows the implementation of this scenario in the 33-point NOLH 
design worksheet. A 129-point NOLH design worksheet can be found in 
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Appendix A. Each FY that is explored has its own worksheet similar to the one 
seen in Figure 26 for FY 2017. PROM-WED provides users with a recruiting 
resource allocation over seven FYs. Therefore, there are seven 33-point NOLFI 
design worksheets and seven 129-point NOLH design worksheets within PROM- 
WED’s data farming wrapper. 


Figure 26. Scenario Inputted into the NOLH Worksheet 



Figure 26 illustrates that each input, whether it be a controllable or 
uncontrollable variable, is tested over 33 levels. Recruiting mission, number of 
recruiters, UE, and relative pay are the variables that are tested over a range of 
values. The lower bound on the range is fed into the “low level” cell, whereas the 
upper bound on the range is fed into the “high level” cell. For the variables that 
remain constant, the low and high values are the same. The “decimals” cell 
refers to the number of significant digits in the decimal place that the NOLH 
algorithm divides the factor into. For example, recruiters, NCO, and OMA 
variables all have a zero in the “decimals” cell since these variables represent 
people, and having a fraction of a person is infeasible. 
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Each row of the worksheet shown in Figure 26 represents a different 
scenario. A subroutine loops over each row of the worksheet and feeds the 
values for each input variable into the legacy PRO model. The subroutine that 
executes 33 design point NOLH excursions can be found in Appendix B. The 
legacy PRO model’s “RunTraditional” macro was adapted to accommodate data 
farming. The modified macro is now referred to as “RunTraditionalS.” 

A 33-design point NOLH design will result in 33 different legacy PRO 
model solutions, and a 129-design point NOLH design will result in 129 different 
legacy PRO model solutions. The NOLH worksheet married with the subroutine 
makes up the data farming wrapper. 

2. Graphical User Interface 

PROM-WED’s GUI makes data farming easily accessible to any PRO 
model user regardless of knowledge or skill in DOE techniques or data farming. 
A snapshot of PROMWED’s GUI is shown in Figure 27. 


Figure 27. PROM-WED’s GUI 
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The variables are categorized as either “Decision Variables” or “Market 
Factors.” A decision variable can either be constrained (“Fixed”) or unconstrained 
(“Floated”). The title “Market Factors” is a blanket category that covers both 
market factors, as well as policy factors, as described in Chapter II. 

A brief description of how a PROM-WED excursion is performed using the 
GUI is now presented. A detailed PROM-WED user manual can be found in 
Appendix C. 

To constrain a decision variable, select the variable of interest and click on 
“Fix DV.” A constrained decision variable can either be fixed as a constant or 
tested over a range of values using the NOLFI algorithm. If the user is interested 
in testing over a range, the desired lower and upper bounds of the range are 
inputted into the “Design of Experiments Table,” as shown in Figure 28. 


Figure 28. Testing a Decision Variable over a Constrained Range 
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Number of recruiters is being tested over a range of 2,500 to 3,500 for each FY of 
this excursion. 
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Each “low level” and “high level” value of the “Design of Experiments 
Table” is linked to a NOLH worksheet. For example, the low and high values for 
FY 17 are linked to the NOLFI worksheet for FY 2017, as shown previously in 
Figure 26. 

A similar procedure is followed for each variable listed in the “Market 
Factors” category. The user must work through each variable in the “Market 
Factors” list, and choose whether it is kept constant (“Fix Value”), or tested over 
a range of values (“Set Range”). The NOLFI DOE is complete once all variables 
listed in the “Market Factors” category are accounted for. Once the NOLFI is fully 
populated, as shown in Figure 29, the user selects the “Run Type,” and the 
number of design points the NOLFI is tested over. Currently PROM-WED has the 
capability to test the traditional run option. Further work is required for the 
capacity run option. 


Figure 29. PROM-WED GUI when NOLH is Fully Populated 
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Selecting the “NOLH RUN” button executes the subroutine to begin 
growing the data. The 33-point design takes approximately two minutes to run on 
a standard modern personal computer (PC), whereas the 129-point design takes 
about five to ten minutes to run. Run times are dependent on factors such as the 
operating system and computational capacity of the computer. The result for 
each PROM-WED scenario is deposited to a worksheet for further analysis. 

3. PROM-WED Decision Support Anaiysis 

In addition to growing PRO model data using data farming, PROM-WED 
provides users with decision support capabilities to analyze the data grown by 
each excursion. PROM-WED offers two decision support capabilities: (1) 
automatically generated analysis and (2) data generated for further analysis 
requiring a statistical software package. In this section, PROM-WED’s decision 
support capabilities are discussed. The focus is on why each type of graph or 
table was chosen. Chapter IV has a detailed discussion dedicated to analyzing 
PROM-WED’s decision support capability. 

a. Automatically Generated Decision Support Capability 

The purpose of PROM-WED’s automatically generated decision support 
analysis is to provide users with a tool capable of providing an at-a-glance 
understanding of the solution space of a completed PROM-WED excursion. 
PROM-WED’s “Decision Support Analysis” for the traditional run option provides 
users with a broad understanding of how variability in decision variables, 
controllable policy changes, and uncontrollable market factors affect the total 
cost of recruiting. Since the traditional run addresses the allocation of resources 
(i.e., the decision variables), the automatically generated decision support 
capability provides at-a-glance insights to decision makers regarding the optimal 
allocation of recruiting resources using the 33-point design. In the next section, 
further insights regarding variable interactions and the effects of the various 
market factors are explored using a commercial statistical software package. 
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In an effort to provide as much relevant information as possible within an 
easily printed worksheet, Figures 30 and 31 show the two pages that comprise 
PROM-WED’s automatically generated decision support capability for the 
traditional run option. 


Figure 30. Traditional Run Decision Support Analysis, Page 1 
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2015 

2016 

2017 

2018 

2019 

2020 

2021 

NCO 

hicti 

40000 

40000 

40000 

40000 

40000 

40000 

40000 

low 

30000 

30000 

30000 

30000 

30000 

30000 

30000 

LRP 

high 

744 

11.22 

11.28 

11.38 

11.43 

11.48 

11.87 

low 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

HSDG 

hic^ 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

low 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

TSC 

hicti 

0.85 

0.85 

0.85 

0.85 

0.85 

0.85 

0.85 

low 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

UE 

hi(ti 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

low 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

Rel Pay 

Nc4i 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

12 

low 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

QMA 

hicti 

1 883304 

1873304 

1863304 

1853304 

1843304 

1833304 

1823304 

low 

1 873304 

1863304 

1853304 

1843304 

1833304 

1823304 

1813304 
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Figure 31. Traditional Run Decision Support Analysis, Page 2 
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The six graphs and one table make up the traditional run’s decision 
support analysis. The purpose of each graph is now explained. 
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Starting in the top left, the “Mean Total Cost of Recruiting over FYDP” 
graph, also shown in Figure 32, shows the resulting mean total cost of recruiting 
for each FY. 

Figure 32. Graph 1: Mean Total Cost of Recruiting over FYDP 
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FY 


The mean for each FY is represented by the blue dots. The red dashed 
lines represent the 95 percent confidence interval for each mean. Where “n” is 
the number of sample points. For example, n = 33 for the 33-point NOLH design, 
and so forth. Here we are treating each observation as an equally likely sample 
of possible recruiting scenarios. The 95 percent confidence intervals for all 
graphs shown in the automatically generated decision support analysis are 
calculated as follows: 

(1) First, the sample standard deviation is calculated: 

2r=i(^t 


The Microsoft Excel formula STDEV.SQ is used in PROM-WED. 
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(2) Next, since each scenario is independent, and it is assumed that 
the sample mean is approximately normally distributed, the margin 
of error at 0.05 significance level is calculated: 


s 

margin of error = z „/2 —=•, 

vn 

where: a = O.OS and 

Zajz is the 100(1 — a/2) percentile of a 
standard nor'mal random variable 


The Microsoft Excel formula CONFIDENCE.NORMQ is used in PROM-WED. 


(3) Finally, the upper and lower confidence bounds are calculated; 

X ± margin of error 

The region between the two red dashed lines represents with 95 percent 
confidence the mean total cost of recruiting is somewhere within this range. 

The second graph “PROM-WED Excursion versus Program of Record,” 
also shown in Figure 33, compares the mean optimal allocation of recruiting 
resources that resulted from the PROM-WED excursion with the program of 
record. 


Figure 33. Graph 2: PROM-WED Excursion versus Program of Record 
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The program of record (POR) is the resource allocation “recorded in the 
current Future Years Defense Program (FYDP) or as updated from the last 
FYDP by approved program documentation” (DAU, n.d.). Within the legacy PRO 
model the POR is fixed for each FY. PROM-WED only reports these fixed 
numbers (i.e., they are the same for each run and are not included in the DOE). 
Each bar of the stacked bar chart is divided into segments that represent the 
amount of resources allocated to each decision variable. A difference between a 
PROM-WED excursion and a POR conveys to an analyst that the Navy should 
consider allocating funds differently to optimize the allocation of recruiting 
resources. These insights support informed decisions such as adjusting the 
number of Sailors assigned to recruiting duty or modifying the amount of 
resources allocated to advertisements and enlistment bonuses. Education 
incentives were not included in the decision support analysis, but can be added if 
the Navy begins to allocate funds towards this resource again. 

The scenario report, shown in Table 10, reports the high and low values of 
each market factor for this PROM-WED excursion. 


Table 10. PROM-WED Scenario Report 



2015 

2016 

2017 

2018 

2019 

2020 

2021 

NCO 

high 

40000 

40000 

40000 

40000 

40000 

40000 

40000 

low 

30000 

30000 

30000 

30000 

30000 

30000 

30000 

LRP 

high 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

low 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

HSDG 

high 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

low 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

TSC 

high 

0.85 

0.85 

0.85 

0.85 

0.85 

0.85 

0.85 

low 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

UE 

high 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

low 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

Rel Pay 

high 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

low 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

QMA 

high 

1883304 

1873304 

1863304 

1853304 

1843304 

1833304 

1823304 

low 

1873304 

1863304 

1853304 

1843304 

1833304 

1823304 

1813304 
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If the high and low values are equal, the market factor is fixed, such as 
NCO in the scenario shown in Table 10. If the market factor is tested over a 
range, the high and low values are not equal, such as assessing the effect of 
varying the percentage of high quality recruits (TSC) from 70 percent to 85 
percent, also shown in Table 10. 

The focus of the second page is on how the decision variables vary. The 
“Total Cost of Recruiting” stacked bar chart shown in Figure 34 indicates how 
much money is allocated to each recruiting resource over a seven FY span. 


Figure 34. Graph 3: Total Cost of Recruiting 
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The following three graphs, shown in Figure 35, represent how deviations 
in controllable and uncontrollable factors affect the amount of resources allocated 
to each decision variable. The blue dots represent the mean for each decision 
variable over each FY, and the red dashed lines represent the 95 percent 
confidence interval about that mean. 
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Figure 35. Graphs 4-6: Decision Variables 
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b. JMP Output 

PROM-WED provides users with output results that are saved as an .xlsx 
file and can be further analyzed using any statistical software package. N1 
analysts use JMP; hence PROM-WED’s output is named “JMP output.” JMP has 
modeling tools, such as partition trees and stepwise regression models, that are 
conducive for testing interactions between multiple variables while quantifying 
and visualizing how they affect the overall solution space. 

PROM-WED’s JMP output is color-coded by variable type, and is 
organized for ease of import into a data analysis package. A snap shot of the 
JMP output for one FY of a 33 design point PROM-WED excursion is shown in 
Figure 36. 
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Figure 36. JMP Output for a 33 Design Point 
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Blue represents the output: Total Cost of Recruiting, green represents the decision 
variables, orange represents policy factors, and red represents the environmental 
factors. 

This thesis uses JMP Pro Version 12 to analyze PROM-WED data using 
six primary techniques: (1) oneway analysis graphs, (2) distributions and 
descriptive statistics, (3) partition trees, (4) stepwise regression models, (5) 
scatterplot matrices, and (6) contour plots (JMP Pro, 2015). The purpose of this 
section is to explain the principal techniques that are used in the analysis section. 
With many of these techniques additional analysis could be done. The analysis 
provided in this research is illustrative of what analysts can do with PROM-WED 
output. 

(1) Oneway Analysis Graphs 

A oneway analysis graph is used to gain a quantifiable understanding of 
the spread of the total cost of recruiting data over each FY. The setup and 
structure of this graph is shown in Figure 37. 
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Figure 37. Oneway Analysis of Total Cost of Recruiting by FY Structure 
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The boxplots that overlay the data represent the information presented in 
the “Quantiles” table. From Figure 37, it is evident that more than 50 percent of 
the data (i.e., the median) is less than the grand mean. The grand mean is 
represented by the horizontal line labeled “mean,” and the median is represented 
by the “50%” label. The median is a useful estimator that provides safety against 
outliers, whereas the mean is highly influenced by extreme values, both high and 
low. 


(2) Distributions and Descriptive Statistics 

Flistograms provide insight regarding the nature of the output data. For 
example. Figure 38 shows that the total cost of recruiting is highly skewed to the 
right. The long tail indicates that there are some particularly large outliers, but the 
majority of the data does not follow this trend. 
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Figure 38. Distribution and Descriptive Statistics Structure 
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(3) Partition Tree 

The setup and structure for a partition tree is shown in Figure 39. 


Figure 39. JMP Partition Tree Structure 
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Adapted from Borozny, 2015, p. 37; Lane, n.d. 


A story can be told from interpreting a partition tree. For instance, the tree 
shown in Figure 39 conveys the following message: 

The mean total cost of recruiting will be approximately $354 million. Since 
advertising is the first child of the parent node, advertising is the dominant 
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decision variable, where 66.5 percent of the variance for the total cost of 
recruiting can be explained. If the cost of advertising remains below $51.4 million, 
then the average cost of recruiting is approximately $325 million. If the cost of 
advertising equals to or exceeds $51.4 million, then the average cost of recruiting 
increases to $432 million. 

(4) Stepwise Regression Model 

Stepwise regression can be used to formulate a prediction model for total 
cost of recruiting, as shown in Figure 40. 


Figure 40. Stepwise Regression Structure 


Beta Estimates 


Regression 

Terms 


Parameter Estimates X 

Term 

Estimate 

Intercept 

150.98005 

1 NCO 

0.025536 

1 Unemployment 

-24.59368 

k Relative Pay 

-550348 

1 (NCO-35000.1)*(Relative Pay-1) 

-0.127001 

1 (NCO-35000.1)’(NCO-35000.1) 

3.0499e« 

^ (Relative Pay-l)*(Relative Pay-1) 

1882.3763 


Std Error 
70.71931 
0.001535 
3.836157 
38.3648 
0.015311 
6.205e-7 
384.6306 


t Ratio 
2.13 
16.64 
-6.41 
-1435 
-8.29 
4.92 
4.89 


Prob> |t| 
0.0348* 
<. 0001 ’ 
<. 0001 * 
<.ooor 
<. 0001 * 
<.ooor 
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The beta estimates and regression terms shown in Figure 40 are used to 
formulate the prediction model shown in Figure 41. 
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Figure 41. Prediction Model for Total Cost of Recruiting Fit Using 

Stepwise Regression 

Prediction Expression 

150.980052850998 
+ 0.02553596931337 *NCO 
+ -24.593679044673* Unemployment 
+ -550.34801858165* Relative Pay 

+ (NCO - 35000.0620155039) * ((RelativePay -1) * -0.1270005027034) 

+ (NCO - 35000.0620155039) * ((NCO - 35000.0620155039) * 0.00000304992539) 
+ (Relative Pay -1) * ((Relative Pay -1) * 1882.37625427676) 


Actual by predicted plots, as shown in Figure 42, demonstrate the 
relationship between the actual data and the model fit using stepwise regression. 
In this case, the closer the points are to the solid red line the better the fit. 


Figure 42. Actual by Predicted Plot. 
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(5) Scatterplot Matrices 

Each panel of the scatterplot matrix in Figure 43 shows the relationship 
between a decision variable, on the x-axis, and the total cost of recruiting, on the 
y-axis. 


Figure 43. Scatterplot Matrix Structure 
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The dark red line within the shaded red region indicates a trend line fit in 
JMP. From these scatterplot matrices, trends can be deduced to help analysts 
further understand the relationships amongst the model’s variables. For example, 
both advertising and EB show a distinct, upward linear trend in relation to the 
total cost of recruiting. The narrow confidence bands around the trend line also 
indicate this is a strong relationship. Whereas, the total number of recruiters has 
only a minor, downward trend. The wider confidence interval around the trend 
line for this plot indicates that the total number of recruiters has minimal effect on 
the total cost of recruiting for this scenario. 

(6) Contour Plots 

Contour plots provide insights similar to the “capture the flag” example 

previously shown in Figure 22, where the multi-level full factorial DOE provides a 

detailed understanding of the solution space. The contour plot in Figure 44 

shows the relation between relative pay and accession mission on the total cost 
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of recruiting. Note that other factors are changing too, so it is important to look for 
broad trends, not local features. 


Figure 44. Contour Plot Structure 

Contour Plot for Total Cost of Recruiting 
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The color variations in Figure 44 represent the total cost of recruiting at 
different combinations of relative pay and new accession mission values. The 
diagonal nature of the plot indicates there is an interaction between relative pay 
and the new accession mission. To minimize the total cost of recruiting, it is 
recommended that the Navy stays within the dark blue regions if the higher 
relative pay is feasible. 

c. Building PROM-WED: Collaboration with Future Users 

To ensure the practicality and future usability of this research, analysts at 
N1 played a critical role in the creation of the PROM-WED tool, specifically in 
regards to the GUI development and the decision support capabilities. A future 
PROM-WED user had hands-on time with the tool to test its limitations and 
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identify potential glitches. Through this meeting, we identified problems with the 
save scenario capability and identified sources of potential confusion that needed 
clarification and were subsequently addressed within the PROM-WED User 
Manual. In addition to the GUI, N1 analysts were involved in the development of 
PROM-WED’s decision support capability. For instance, the JMP output color¬ 
coding and the scenario report were added to the automatically generated 
decision support capability based on feedback from N1 analysts. 

As with any new tool, it may take time for N1 analysts to become 
accustomed to using PROM-WED. For example, it was requested that a graph 
be added to the automatically generated decision support capability that 
displayed how unemployment rate effects the total cost of recruiting over each 
FY. An example of this graph is shown in Figure 45. The parameter inputs for the 
PROM-WED excursions shown in Figures 45 and 46 can be found in Appendix 
D. 


Figure 45. Effect of Unemployment Rate on Total Cost of Recruiting 


380 


—•—2015 

—•—2016 

—•—2017 

—•—2018 

—•—2019 

-•-2020 

—•—2021 



As expected, when the unemployment rate is low, the cost of recruiting is 
high, and as unemployment rate increases the cost of recruiting decreases. 

However, PROM-WED is capable of testing uncertainties in multiple variables, 
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not just one. When more than one variable is tested over a range, the graph 
becomes difficult to interpret. For example, Figure 46 is a PROM-WED excursion 
with the same input parameters as the PROM-WED excursion shown in Figure 
45 except the number of recruiters is bounded from 2,500 to 3,500, instead of 
fixed at 3,913 as shown in Figure 45. 

Figure 46. Effect of Varying Unemployment Rate and Number of 
Recruiters on Total Cost of Recruiting 
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From Figure 46, it is evident that bounding the number of recruiters does 
affect the total cost of recruiting. Flowever, it is difficult to discern any valuable 
insights from Figure 46 regarding the interactions that are occurring between the 
varied number of recruiters and the unemployment rate on the total cost of 
recruiting. This example only varied two variables, whereas excursions that are 
explored in the next section vary up to six variables. 

Examples such as this one demonstrate that through the implementation 
of DOE techniques, PROM-WED delivers results that provide valuable insights 
into the optimal allocation of recruiting resources. Flowever, this added capability 
challenges the legacy analysis methods used to study legacy PRO model 
outputs. 
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IV. ANALYSIS 


Through two test case examples, this chapter showcases PROM-WED’s 
ability to deliver comprehensive insights into the optimal allocation of recruiting 
resources. The chapter begins with the introduction of the two test case 
examples, referred to as Test Case 1 and Test Case 2. These examples are first 
analyzed through PROM-WED’s automatically generated decision support 
capabilities, and further explored using an array of statistical modeling and 
graphing methods in JMP. Finally, a modified version of Test Case 1 is used to 
compare the number of runs required for a full factorial DOE to the NOLH 
designs used in PROM-WED. 

A. TEST CASES 

To demonstrate PROM-WED’s capabilities, N1 formulated three separate 
scenarios to model best case, worst case, and most likely situations for Navy 
recruiting. These scenarios are found in Appendix E. Rather than running three 
separate scenarios, PROM-WED can test this broad spectrum of possibilities and 
uncertainties using a single data farming run. 

Test Case 1 explores uncertainties in economically driven market factors 
(i.e., relative pay and unemployment rate). Test Case 2 adds two additional 
degrees of uncertainty to Test Case 1 in the form of policy factor changes (i.e., 
OMA and recruit quality). All market factors not listed in the tables remain at their 
default values from the legacy PRO model. The scenario reports for each run are 
available in Appendix D. 

a. Test Case 1 

Test Case 1 covers a broad spectrum of economic uncertainties that 
represent best case, worst case, and most likely scenarios for Navy recruiting. 
For example, a low unemployment rate, relative pay favoring the civilian sector, 
and a high recruiting accession mission are challenging conditions for Navy 
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recruiting. On the other hand, a high unemployment rate, relative pay favoring 
the military, and a low recruiting accession mission would be favorable conditions 
for Navy recruiting. The input values for Test Case 1 are shown in Table 11 and 
can be used to answer a question such as: 

What is the optimal allocation of recruiting resources that is robust to a 
broad range of economic uncertainties? 


Table 11. Test Case 1 Input Variables 


Variable Type 

Variable Name 

Value Low 

Value High 

Decision Variable 

Recruiters 

2,500 recruiters 

3,500 recruiters 

Market Factor 

Unemployment Rate 

4.0% 

8.0% 

Market Factor 

Relative Pay 

0.80 

1.20 

Policy Factor 

Recruiting Mission (NCO) 

30,000 recruits 

40,000 recruits 


For additional scenario details, refer to Appendix D. 

b. Test Case 2 

Test Case 2 maintains the foundation of Test Case 1, but adds the effects 
of varying two policy factors: (1) percentage of high quality recruits, and (2) 
qualified military available. Test Case 2’s input variables are shown in Tables 12 
and 13, and can be used to answer a question such as: 

What is the optimal allocation of recruiting resources if the Navy desires to 
increase the percentage of high quality recruits from 70 percent to 85 percent? 
Due to uncertainties in the current fiscal environment, the unemployment rate 
may fluctuate between 4 to 8 percent, and the ratio of relative pay may vary 
between 0.8 and 1.2. In addition, since marijuana has been legalized for 
recreational use in many states nationwide, drug-use amongst 18-24 year-olds is 
expected to increase. An increase in drug-use amongst this age group means 
fewer young adults qualify for military service. Test Case 2 models the effect of 
an annual decrease of 10,000 qualified military available due to pre-service drug- 
use. 
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Table 12. Test Case 2 Input Variables 


Variable Type 

Variable Name 

Value Low 


Decision Variable 

Production Recruiters 

2,500 recruiters 

3,500 recruiters 

Market Factor 

Unemployment Rate (UE) 

4.0% 

8.0% 

Market Factor 

Percentage of Fligh Quality 
Recruits (TSC l-lll) 

70% 

85% 

Market Factor 

Relative Pay 

0.8 

1.2 

Market Factor 

Qualified Military Available 
(QMA) 

*See Table 13 

Policy Factor 

Recruiting Mission (NCQ) 

30,000 recruits 

40,000 recruits 


Since Test Case 2 models the cumulative effects that the legalization of 
marijuana may have on the nation’s QMA, the input values for QMA will decrease 
by 10,000 each FY. The QMA input values for Test Case 2 are shown in Table 
13. 


Table 13. Traditional Run 2 QMA Input Values 


FY 

QMA Value Low 

QMA Value High 

2015 

1,873,304 

1,883,304 

2016 

1,863,304 

1,873,304 

2017 

1,853,304 

1,863,304 

2018 

1,843,304 

1,853,304 

2019 

1,833,304 

1,843,304 

2020 

1,823,304 

1,833,304 

2021 

1,813,304 

1,823,304 


For more information regarding Test Case 2 parameter inputs, refer to 
Appendix D. 

B. DECISION SUPPORT ANALYSIS 

As explained in Chapter III, PRQM-WED automatically generates a 
selection of graphs to provide decision-makers with an “at-a-glance” 
understanding of the solution space. The 33-point design grows a sufficient 
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amount of data for basic statistical analysis in under two minutes. Since the 
purpose of the decision support analysis is to provide a quick understanding of 
the solution space, only the 33-point NOLH design is analyzed in this section. 
This type of analysis would be appropriate for testing excursions during a time 
constrained meeting, working group, or whenever basic analysis needs to be 
generated quickly. The 129-point NOLH grows more data, requiring a longer run 
time and more time is needed for adequate analysis. The 129-point NOLH is 
used in the JMP analysis section. 

1. Test Case 1 

Some major insights that are gained from Test Case Ts automatically 
generated decision support capability are now discussed. Figure 47 
demonstrates that in an uncertain economic environment, the mean total cost of 
recruiting in FY 2017 will be within $350 million to $450 million, with 95 percent 
confidence. 

Figure 47. Test Case 1: Mean Total Cost of Recruiting over FYDP 
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Figure 48 indicates that on average, the optimal cost of recruiting for each 
FY complements the program of record (POR) budget estimate. 
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Figure 48. Test Case 1: PROM-WED Excursion versus POR 
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The graph shown in Figure 48 can also inform decisions to redistribute 
funds to optimize the allocation of resources to advertisements, enlistment 
bonuses, and recruiters. For example, in the same graph, now labeled Figure 49, 
informed recommendations can be made to distribute resources differently in 
order to optimize the allocation of recruiting resources. 


Figure 49. Test Case 1: PROM-WED versus POR 
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Dependent upon FY, if the dark blue bar is higher for POR than PROM- 
WED, this indicates that in order to optimize the allocation of recruiting 
resources, less resources need to be allocated to recruiters. Less funding 
allocated to recruiters means less recruiters are required in the field. The same 
convention goes for enlistment bonuses and advertisements. For example, in FY 
2020 less funds should be allocated to recruiting and more funds should be 
allocated to enlistment bonuses and advertisements. 

Figure 50 shows that the optimal allocation of recruiting resources 
appears to sustain a consistent trend amongst the seven FYs with only a minor 
upward trend, most likely due to inflation rates. 


Figure 50. Test Case 1: Resource Allocation Breakdown 



Insights gained through Figures 48 and 50 indicate there is evidence to 
believe that the total cost of recruiting is robust to uncertainties in the economic 
environment. Flowever, to optimize the allocation of resources, more resources 
need to be allocated to enlistment bonuses and advertisements, as shown 
previously in Figure 49. 
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Figure 51 indicates that, with 95 percent confidence, the optimal allocation 
of resources to advertising over the seven FY span is consistently maintained 
within the range of approximately $40 million to $80 million. 


Figure 51. Test Case 1: Advertising 
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Similar to the insights gained from Figure 51, Figure 52 demonstrates that 
with 95 percent confidence, the optimal allocation of resources to enlistment 
bonuses over the seven FY span consistently maintains a range of $50 million to 
$110 million. 


Figure 52. Test Case 1: Enlistment Bonuses 
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Test Case 2 


The effects of a shrinking QMA pool and an increased requirement for 
recruit quality is analyzed through the comparison of Test Case 1 and Test Case 
2 . 

From Figure 53, there is evidence to believe that the Navy can expect the 
total cost of recruiting to increase by approximately $50 million as the need for 
high quality recruits increases, and the QMA pool shrinks. Without these policy 
influences, the 95 percent confidence interval increased from $350 million to 
$450 million in Test Case 1, to approximately $400 million to $500 million in Test 
Case 2. 


Figure 5 

600 

00 

c 

5 500 

3 

>3. Test Case 2: Mean Total Cost of Recruiting over FYDP 








> -< 

) - Confidence 

Total Cost of Rect 
(Millions) 

ro ^ 

58888 

►-< 

> 






r 

• Mean 









- Confidence 



















2015 2016 

2017 

2018 2019 2020 2021 


FY 


Figure 54 indicates that the total cost of recruiting is expected to exceed 
the program of record for every FY. 
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Figure 54. Test Case 2: PROM-WED Excursion versus POR 
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To optimize the allocation of recruiting resources, there appears to be a 
consistent trend amongst all seven FYs that an excess of resources was 
allocated to recruiters in the POR, while more resources should be allocated to 
advertisements and enlistment bonuses instead. 

Due to the addition of QMA uncertainties and recruit quality policy 
changes. Figure 55 indicates that the average cost of recruiting is expected to 
increase by approximately $50 million over the seven FY span. This is a 
noticeable increase over the trend previously shown for Test Case 1 in Figure 50. 
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Figure 55. Test Case 2: Resource Allocation Breakdown 



Figures 56 and 57 juxtapose results for advertisement and enlistment 
bonus resource allocations for Test Case 1 and Test Case 2. 


Figure 56. Resources Allocated to Advertising 
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Left: Test Case 1, right: Test Case 2. 


Figure 56 indicates that the cost of advertising will increase by an average 
of approximately $10 million each FY. 
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Figure 57. Resources Allocated to Enlistment Bonuses 
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Figure 57 indicates a similar trend for enlistment bonuses. These graphs 
show that an additional $20 million will be required for enlistment bonuses each 
FY due to the addition of QMA uncertainties and proposed recruit quality 
changes. 

C. GRAPHICAL AND STATISTICAL ANALYSIS IN JMP 

Valuable insights can be found through analyzing variable interactions and 
uncertainties that shape the robust solution space. Flowever, analyzing and 
visualizing variable interactions in Microsoft Excel is difficult due to the software’s 
limited statistical capability. Analysts will need to use a statistical software 
package to take full advantage of the data grown by PROM-WED. Test Case 1 
and Test Case 2 are now analyzed using JMP. 

1. Test Case 1 

To gain an initial understanding of the data, Figure 58 shows the spread of 
data and provides quantile metrics for each FY. From Figure 58, it is evident that 
over 50 percent of the data, indicated by the median, generated for each FY is 
below the grand mean total cost of recruiting for each FY. 
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Figure 58. Total Recruiting By FY with Interquartile Ranges 
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2015 
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309.1051 

349.9401 

421.2093 

521.9235 

959.1079 

2016 

258.138 

289.8711 

315.9455 

35639 

428.4688 

529.2777 

968.0586 

2017 

261.6095 

293.5959 

320.1883 

361.2135 

433.1215 

533.85 

974.16% 

2018 

265.1709 

297.4278 

324.5511 

365.861 

437.8086 

538.5172 

980.3789 

2019 

268.7332 

301.3375 

329.0443 

370.5412 

442.3917 

543.1901 

986.5974 

2020 

272.8857 

305.6795 

334.2964 

375.5285 

447.5582 

548.4681 

993.3847 
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277.2888 

310.3785 

339.8189 
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554.0022 

100043 


MAX 



The outliers in Figure 58, highlighted below in Figure 59, are worth 
examining further to determine if there is a common cause for the four unusual 
data points. Using JMP, the highlighted sixteen data points are lassoed (i.e., 
selected) to reveal that the 78^^^, 80*^^, 88'*^, and runs for each scenario 

caused these results over each FY. The run numbers represent four of the 129 
different scenarios built using PROM-WED’s 129-point NOLH DOE. Since each 
FY uses the same NOLFI DOE, the 80*'^ run for each FY of Test Case 1 is tested 
over the same input market factors and number of recruiters. The same 
convention applies for the 78*'^, 88*^^, and 96'*^ runs as well. 
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Figure 59. Outliers for Each FY 

* Oneway Analysis of Total Cost of Recruiting By FY 

1000 

900 
a^ 

I 800 

1 700 

2 600 

i 300 
o 

400 
300 
200 


The upward trend occurs due to yearly changes, such as inflation rates, 
elasticities, or input values from the legacy PRO model. The input variables for 
each run highlighted in Figure 59 are shown in Table 14. 
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Table 14. Test Case 1 Input Variables for Output Outliers 


Run # 



78 

80 

88 

96 

Recruiters 

2547 

2523 

2727 

2789 

NCO 

39531 

39688 

39453 

39609 

UE 

7.0 

5.9 

5.1 

5.1 

Relative Pay 

0.84375 

0.878125 

0.85 

0.853125 


The most extreme total cost of recruiting outlier, resulting from the 80*'^ 
run, tested an excursion where the Navy had a very low number of recruiters in 
the field (just over 2,500 recruiters), the new accession mission was extremely 
high (almost at 40,000 new recruits), unemployment rate was mediocre, and the 
relative pay favored the civilian sector. The other three runs also showed similar 
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trends where high accession missions, with a low number of recruiters in the 
field, and relative pay highly favoring the civilian sector resulted in unusually high 
expected recruiting costs. Identifying these costly outliers can help N1 analysts 
make informed recommendations to avoid situations like the 80**^ run by 
preemptively increasing the number of recruiters in the field. 

To gain additional situational awareness of the data, the distributions and 
descriptive statistics for each decision variable are explored. Figure 60 shows the 
histogram of the distribution for total cost of recruiting over one FY of the PROM- 
WED excursion. Flistogram and descriptive statistics for resourcing to 
advertisements and enlistment bonuses can be found in Appendix F. 


Figure 60. Flistogram and Descriptive Statistics for Total Cost of 

Recruiting Distribution for FY 2017 
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2.5% 

276.82095 



0.5% 

261.6095 



0.0% minimum 

261.6095 




The histogram indicates that the distribution is highly skewed to the right. 
As well, the four data points that appear in the far right side of the histogram 
again represent runs 78, 80, 88, and 96. 

Partition trees were used to understand how variable interactions and 
economic uncertainties affected the solution space. The partition trees in Figures 
61 and 62 take into consideration the influence of each decision variable on the 
total cost of recruiting. Figure 61 shows the first split of the partition tree for total 
cost of recruiting, specifically for FY 2017. 
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Figure 61. First Split of Total Cost of Recruiting Partition Tree 



Figure 61 indicates that resourcing to advertising is the most influential 
predictor of the total cost of recruiting. 61.9 percent of the variance for the total 
cost of recruiting can be explained based on the first split of the partition tree. 
When less than $139 million is allocated to advertising, then the mean total cost 
of recruiting will be approximately $371 million. If more than $139 million is 
allocated to advertising, then the mean total cost of recruiting will increase to 
almost $755 million. 

Figure 62 shows the next split of the partition tree shown in Figure 61. 
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Figure 62. Second Split of Total Cost of Recruiting Partition Tree 


▼ Partition for Total Cost of Recruiting 



RSquare 

RMSE 

N 

Number 
of Splits 

AICc 

Training 

0.824 

52.07929 

129 

2 

139422 

Validation 

0.815 

53.818414 

774 




All Rows 

Count 

Mean 

Std Dev 

129 LogWorth Difference 
398.24689 57.405143 38346 

124.63855 


Advertising <139.1744 

Advertising >=139.1744 

Count 

Mean 

Std Dev 

120 LogWorth Difference 
371.49385 78.164702 125.276 

69.85107 

Count 

Mean 

Std Dev 

9 

754.95403 

148.6531 


> Candidates 


Ad V erti si n g < 5 5 . 0222 
Count 82 

Mean 331.82312 
Std Dev 33.803821 

Advertising >=55.0222 
Count 38 

Mean 457.09912 

Std Dev 46.655738 

> Candidates 

> Candidates 


The second split of the partition tree indicates that resourcing to 
advertising is identified again as the dominant predictor of the total cost of 
recruiting. Based on this split, over 80 percent of variance in the total cost of 
recruiting is explained. Repeated splitting of the same factor, in this case 
resources allocated to advertising, indicates regression may be a more 
informative analysis technique. 

Next, a partition tree is used to understand which market factors most 
influence advertising. Figure 63 shows the parent and first child node of the 
partition tree for advertising. 
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Figure 63. Parent and First Child Node of Partition Tree for Advertising 


Partition for Advertising 



The partition tree in Figure 63 indicates that the recruiting accession 
mission is the most influential factor on the cost of advertising. The relatively 
small R-squared value indicates that a single split on accession mission explains 
only 39.6 percent of variance. In particular, if the accession mission is below 
38,750 new recruits, the mean resourcing towards advertising is approximately 
$44.4 million. If the accession mission exceeds 38,750 new recruits, then the 
mean resourcing to advertising increases by over $100 million, to $148.6 million. 

Following seven additional splits, as shown in Figure 64, it is evident that 
the resourcing of funds to advertising is influenced by many factors, to include: 
the new accession mission, relative pay, and to a small extent, the 
unemployment rate. Since it took seven splits to surpass the 80 percent R- 
squared threshold, it is evident that these three factors influence the resourcing 
of funds to advertising, but none of them particularly dominate. 
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Figure 64. Partition Tree for Advertising After Seven Splits 


▼ Partition for Advertising 
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Stepwise regression is another method used to gain insights into how 
variables influence the solution space. Using stepwise regression with some 
manual judgement, the parameter estimates shown in Figure 65 are used to 
formulate the prediction model for the total cost of recruiting, shown in Figure 66. 


Figure 65. Stepwise Regression for Total Cost of Recruiting 


Parameter Estimates 

Term 

Estimate 

Std Error 

t Ratio 

Prob>|t| 

Intercept 

150.98005 

70.71931 

2.13 

0.0348* 

NCO 

0.025536 

0.001535 

16.64 

<.0001* 

Unemployment 

-24.59368 

3.836157 

-6.41 

<.0001* 

Relative Pay 

-550348 

38.3648 

-1435 

<.0001* 

{NCO-35000,l)*(Relative Pay-1) 

-0.127001 

0.015311 

-8.29 

<.0001* 

{NCO-35000,1)*(NCO-35000.1) 

3.0499e-6 

6.205e-7 

4.92 

<.0001* 

(Relative Pay-l)*(Relative Pay-1) 

1882.3763 

384.6306 

4.89 

<.0001* 


The stepwise regression model exhibits how the NOLFI DOE allows for 
non-linear relationships and interactions. 
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Figure 66. Test Case 1, Prediction model for Total Cost of Recruiting 

Prediction Expression 

150.980052850998 
+ 0.02553596931337* NCO 
+ -24.593679044673* Unemployment 
+ -550.34801858165* Relative Pay 

+ (NCO - 35000.0620155039) * ((Relative Pay -1) * -0.1270005027034) 

+ (NCO - 35000.0620155039) * ((NCO - 35000.0620155039) * 0.00000304992539) 
+ (Relative Pay -1) * ((Relative Pay -1) * 1882.37625427676] 


The prediction model for total cost of recruiting indicates that the new 
accession mission and relative pay interact to effect the total cost of recruiting. 
The new accession mission and relative pay both exhibit a non-linear behavior as 
evidence by their polynomial to degree two interactions. This relationship can 
also be visualized in the prediction profiler shown in Figure 67. 


Figure 67. Prediction Profiler for Varying Factors in Test Case 1 


Prediction Profiler 



NCO Unemployment Pay 


The prediction profiles for the new accession mission and relative pay 
shown in Figure 67 demonstrate their quadratic nature. 
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Next, the summary of fit for the regression model shown is shown in 
Figure 68. 


Figure 68. Summary of Fit for Total Cost of Recruiting Prediction Model 


Summary of Fit 

RSquare 

0.842252 

RSquare Adj 

0,834494 

Root Mean Square Error 

50.70598 

Mean of Response 

398.2469 

Observations (or Sum Wgts) 

129 


This model explains over 84 percent of the variance of the total cost of 
recruiting for FY 2017. 

To visualize a comparison of this model to actual FY 2017 data, the actual 
versus predicted plot is shown in Figure 69. 


Figure 69. Test Casel, Actual by Predicted Plot for Total Cost of 

Recruiting 


Actual by Predicted Plot 



200 300 400 500 600 700 800 900 1000 
Total Cost of Recaiiting Predicted P<.0001 
RSq=0.84 RMSE=50.706 


Figure 70 highlights the outlying points. Once again, runs 78, 80, 88 and 
96 appear to be outliers. 
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Figure 70. Test Case1, Actual by Predicted Plot with Outliers 


Actual by Predicted Plot 



Total Cost of Recruiting Predicted P<.0001 
RSq=0.84 RMSE=50.706 

Since the new accession mission, unemployment rate, and relative pay 
drive advertising resourcing, six scatterplot matrices, shown in Figure 71, help 
analysts visualize trends amongst these factors against the total cost of recruiting 
and the resourcing of funds to advertising. As before, we plot the response (in 
this case, total cost of recruiting and advertising costs) against new accession 
mission, unemployment rate, and relative pay. The values for the new accession 
mission, unemployment rate, and relative pay come from the NOLFI DOE. Other 
factors such as allocated funds to EB and the number of recruiters in the field, 
are also changing (EB is being optimized, while number of recruiters comes from 
the NOLH DOE). Therefore, the trends in these scatterplot matrices should be 
considered through the lens of a broad picture, not localized trends. 
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Figure 71. Economic Factor Trends on Recruiting Resource Allocation 
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The two scatterplot matrices for NCO versus advertising and NCO versus 
total cost of recruiting both indicate an upward trend, where a higher accession 
mission correlates with more resources allocated toward advertising and a higher 
total cost of recruiting. Both of the unemployment rate graphs show minor signs 
of a downward trend indicating that the cost of recruiting and the allocation of 
resources to advertising decreases, as the unemployment rate increases. Lastly, 
the relative pay versus advertising and relative pay versus total cost of recruiting 
graphs also indicate a trend. As the relative pay begins to increase, meaning 
wages favor the military over the civilian sector, resourcing towards advertising 
begins to decrease and the total cost of recruiting also decreases. 

The four outlying points from runs 78, 80, 88 and 96 are present in these 
scatterplots as well. Figure 72 highlights results from these four runs. Once 
again, they appear to be outliers within each scatterplot. 
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Figure 72. Economic Factor Trends on Recruiting Resource Allocation 

with Outliers 


'' Scatterplot Matrix 




Variable interactions can also be shown in a three-dimensional manner 
using contour plots. The contour plot in Figure 73 represents the interaction 
between relative pay and accession mission on the total cost of recruiting. 
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Figure 73. Three-Dimensional Representation of Relative Pay and NCO 

Effects on the Total Cost of Recruiting 


Contour Plot for Total Cost of Recruiting 
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The diagonal color transition indicates the presence of interactions. The 
red region, in the upper left portion of the plot represents the interaction between 
relative pay and new accession mission that result in the most costly conditions 
for Navy recruiting. This region represents when wages favor the civilian sector 
and the accession mission is high. The dark blue area represents the opposite 
conditions, where the total cost of recruiting is the lowest when the accession 
mission is relatively low and relative pay favors the military. 

The contour plot shown in Figure 74 illustrates the relationship between 
relative pay and recruit accession mission on resources allocated toward 
advertising. 
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Figure 74. Three-Dimensional Representation of Relative Pay and NCO 

Effects Resourcing to Advertising 


Contour Plot for Advertising 
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Figure 74, which also exhibits a diagonal nature, indicates that nearly half 
of the solution space supports a low advertising budget, represented by the dark 
blue region. The cost of advertising substantially increases when relative pay 
favors the civilian sector and the accession mission is high, represented by the 
red region. Once relative pay exceeds approximately 1.00, changes in the new 
accession mission have little to no effect on the amount of resources allocated to 
advertising. 

2. Test Case 2 

To further understand how the addition of two policy uncertainties affect 
the optimal allocation of recruiting resources. Test Case 2 is explored using JMP. 
As in the previous section, emphasis is placed on comparing insights gained that 
may distinguish Test Case 2 from Test Case 1. 
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To gain an initial understanding of the data, Figure 75 shows the span of 
possible costs of recruiting over each FY. 


Figure 75. Total Cost of Recruiting by FY with Interquartile Ranges 


Oneway Analysis of Total Cost of Recruiting By FY 
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2016 
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2018 

FY 

2019 

2020 

2021 

1 Quantiles 

Level 

Minimum 

10% 

25% 

Median 

75% 

90% 

Maximum 

2015 

253.7634 

296.5226 

318.9388 

378.5047 

467.3705 

571.5925 

1019.142 

2016 

260.1305 

303.2767 

325.8074 

385.8551 

475.3449 

580.2986 

1030.877 

2017 

263.6967 

307.3261 

329.9298 

390.5322 

480.6594 

586.3599 

1039.902 

2018 

267.3544 

311.4732 

334.3055 

395.3136 

486.0845 

592.5386 

1049.072 

2019 

271.0144 

315.6291 

338.7466 

400.1104 

491.531 

598.7463 

1058.296 

2020 

275266 

320.347 

343.8861 

405.5857 

497.6674 

605.6559 

1068.192 

2021 

279.7699 

325.1176 

349.2917 

411.3294 

504.0784 

612.8481 

1078.395 


It is evident that the grand mean total cost of recruiting increased by 
almost $50 million in comparison to Test Case 1’s grand mean total cost of 
recruiting shown previously in Figure 58. As well, Figure 76 shows that runs 80, 
88, and 96 model conditions result in unusually high expected recruiting costs. 
From Figure 76, it is difficult to distinguish the difference between runs 80 and 
88 . 
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Figure 76. Test Case 2 Outliers 
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The input values that are common for each run over the seven FY 
excursion are shown in Table 15. The annual decrease in QMA values is 
explored later in this section. 


Table 15. Test Case 2 Outlier Input Values 


Run # 



80 

88 

96 

Recruiters 

2523 

2727 

2789 

NCO 

39688 

39453 

39609 

UE 

5.9 

5.1 

5.1 

Relative Pay 

0.878125 

0.85 

0.853125 

TSC l-IIIA 

0.83 

0.71 

0.85 


In comparison to Test Case 1, where the 96*^^ run was the “least extreme 
of the extreme” values, the 96'*^ run for Test Case 2 consistently modeled the 
“most extreme of the extreme” values. This indicates that the increase in recruit 
quality and annual decrease in QMA affected the optimal allocation of recruiting 


resources. 
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Additional insights can be gained by comparing the quantile metrics for 
both Test Cases. The quantile charts for both test cases are shown in Figure 77. 


Figure 77. Test Case 1 and Test Case 2 Quantile Charts 


^ Quantiles 
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959.1079 
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356.89 

428.4688 
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533.85 
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297.4278 

324.5511 
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437.8086 

538.5172 
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2019 

268.7332 

301.3375 

329.0443 

370.5412 

442.3917 
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986.5974 

2020 

272.8857 

305.6795 

334.2964 

375.5285 

447.5582 

548.4681 

993.3847 

2021 

277.2888 

310.3785 

339.8189 

380.7692 

453.0851 

554.0022 

100043 

J Quantiles 

Level 
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10% 

25% 

Median 

75% 

90% 

Maximum 

2015 

253.7634 

296.5226 

318.9388 

378.5047 

467.3705 

571.5925 

1019.142 

2016 

260.1305 

303.2767 

325.8074 

385.8551 

475.3449 

580.2986 

1030.877 

2017 

263.6967 

307.3261 

329.9298 

390.5322 

480.6594 

586.3599 

1039.902 

2018 

267.3544 

311.4732 

334.3055 

395.3136 

486.0845 

592.5386 

1049.072 

2019 

271.0144 

315.6291 

338.7466 

400.1104 

491.531 

598.7463 

1058.296 

2020 

275266 

320347 

343.8861 

405.5857 

497.6674 

605.6559 

1068.192 

2021 

279.7699 

325.1176 

349.2917 

411.3294 

504.0784 

612.8481 

1078.395 


Top: Test Case 1; bottom: Test Case 2. 


Figure 77 helps inform analysts that over each FY, Test Case 2 requires 
more resources than Test Case 1. The differences between the minimum values 
for each Test Case are approximately $2 million across each FY. This spread 
can increase upwards of $70 million when comparing differences between 
maximum values of both cases. As well, the interquartile ranges, the difference 
between the 25th and 75th quartiles which represent 50 percent of the data, is 
approximately $113 million for Test Case 1 and increases to approximately $150 
million for Test Case 2. 

Figure 78 juxtaposes the distributions and descriptive statistics for Test 
Case 1 and Test Case 2. 
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Figure 78. Distributions and Descriptive Statistics for Test Cases 1 and 

2 


Total Cost of Recruiting 

Kno)—I-.. 



300 400 500 600 700 800 900 1000 


Quantiles Summary Statistics 


100 j0% 


974 1696 

.Mean 

398 24689 

99.5% 


974 1696 

StdOev 

124 63855 

97.5% 


875 44575 

Std En Mean 

10.973812 

90.0% 


53385 

Upper 95% Mean 

419960)5 

75.0% 

qu«rtil« 

433.12145 

Louver 95% Mean 

376.53333 

50.0% 

median 

3612135 

N 

129 

25.0% 

quartile 

320.18825 



10.0% 


293 5959 



2.5% 


276.82095 



0.5% 


261 6095 



0.0% 

inoimum 

2616095 




w 

Total Cost of Recruiting 







Quantiles 

^ ' Summary Statistics 


Mlo 


* M t • 


n 


h- 


300 400 500 600 700 800 900 1000 


1000% 

maximum 

1039.9023 

Mean 

428.36779 

99.5% 


1039.9023 

Std Dev 

135.96931 

97.5% 


883.5796 

Std Err Mean 

11.97143 

90.0% 


586.3599 

Upper 95% Mean 

452.05531 

75.0% 

quartile 

480.6594 

Lower 95% Mean 

404.68027 

50.0% 

median 

390.5322 

N 

129 

25.0% 

quartile 

329.9298 



10.0% 


307.3261 



2.5% 


286.1116 



0.5% 


263.6967 



0.0% 

minimum 

263.6967 




Top: Test Case 1; bottom: Test Case 2. 


Figure 78 indicates that the distributicn cf recruiting ccsts fcr Test Case 2 
is pcsitively skewed with a Icng right tail, as was the case fcr Test Case 1. Test 
Case 2’s right tail appears tc be wider than what was seen fcr Test Case 1. A 
wider right tail indicates that Test Case 2 prcduced mere expensive combinatiens 
cf recruiting rescurces, also referred to as outliers, in comparison to Test Case 1. 

When comparing the mean and median values for each Test Case, the 
differences between the mean and median values for Test Case 1 and Test Case 
2 are approximately equal, at $37 million and $38 million, respectively. This 
suggests that the mean total cost of recruiting is heavily influenced by the 
outliers, but even with the presence of more outliers in Test Case 2, the 
differences between the mean and median estimators are negligible. 
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As in Test Case 1, a partition tree identifies advertising as the most 
influential decision variable for Test Case 2. Figure 79 shows that 62.1 percent of 
variance in the total cost of recruiting can be explained from a split on 
advertising. 


Figure 79. First Split for Test Case 2 


Partition for Total Cost of Recruiting 



Following four splits, Figure 80 indicates that when the R-squared value 
exceeds .80, and even .92 in this case, advertising continues to dominate the 
partition tree. 
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Figure 80. Test Case 2 Following Four Splits 


'' Partition for Totai Cost of Recruiting 

Number 

RSquare RMSE N ofSpIKs AICc 


Training 

Validation 

0.924 37.450561 129 4 1313.51 

0.904 42.368977 774 




As in Test Case 1, repeated splitting on advertising indicates regression 
as an appropriate technique for further analysis. 

Partition trees are also constructed to determine how uncertainties in QMA 
and a policy change in recruit quality could affect resourcing to advertising. Flere 
advertising is the response variable and we are investigating which factors 
influence advertising. Figure 81 shows the first split of this tree. 
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Figure 81. Test Case 2: First Split of Advertising 


Partition for Advertising 



Similar to results found in Test Case 1, the new accession mission is 
identified as the dominant factor, but it maintains a low variance explained at 
37.8 percent. Following three more splits, the R-squared value doubled. The 
resulting partition tree is shown in Figure 82. 
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Figure 82. Test Case 2: Partition Tree for Advertising 


Partition for Advertising 



RSquare 

RMSE 

N 

Number 
of Splits 

AICc 

Training 

0.765 

29.121039 

129 

4 

1248.61 

Validation 

0.764 

29.404593 

774 




All Rows 

Count 
Mean 
Std Dev 

129 LogWorth Differersce 
71.101741 20.071571 109.121 

60.307798 


NCO>=38750 

Count 

Mean 

Std Dev 

17 LogWorth Difference 
165.8425 9.8800808 162.576 

88.815561 


NCO<38750 

Count 
Mean 
Std Dev 

112 LogWorth DiffererKe 
56.721447 16.796722 48.7264 

38.388607 


Relative Pay>=0.953125 

Count 72 LogWorth Differeixe 

Mean 39.319154 14.051149 32.69 

Std Dev 23.047752 

Relative Pay <0.953125 
Count 40 

Mean 88.045575 

Std Dev 40.765568 


> Candidates 


Relative Pay>=0.9125 
Count 11 

Mean 108.46275 
Std Dev 35.890963 

Relative Pay <0.9125 
Count 6 

Mean 271.0387 

Std Dev 46.293454 

> Candidates 

> Candidates 


NCO<36250 

Count 49 

Mean 28.876531 
Std Dev 12.654269 

NCO>=36250 

Count 23 

Mean 61.566483 
Std Dev 24.589152 

Candidates 

> Candidates 


Figure 82 indicates that the new accession mission and relative pay 
predominately drive the allocation of resources to advertising. It is interesting to 
note that neither QMA nor recruit quality appear in this partition tree. This 
suggests that they have a minimal, if any, influence on advertising resources. 

Again, stepwise regression with manual judgement, is used to formulate a 
model to predict the total cost of recruiting for Test Case 2. The parameter 
estimates used to formulate the prediction model are shown in Figure 83. 
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Figure 83. Test Case 2, Stepwise Regression for Total Cost of 

Recruiting 


Parameter Estimates 

Term Estimate Std Error t Ratio Prob>|t| 

Intercept 


NCO 
TSCI-mA 
Unemployment 
Relative Pay 

(NCO-35000.1)*(Unemployment-6.0031) 
(NCO-35000,l)*(Relative Pay-1) 
(TSCI-IIIA-0.77504)*(RelativePay-l) 
(Unemployment-6.0031)*(Relative Pay-1) 
(NCO-35000,1)*(NCO-35000.1) 

(Relative Pay-l)*(Relative Pay-1) 


-126.5045 

74.71878 

-1.69 

0.0931 

0.0266897 

0.001086 

24.57 

<.ooor 

562.69782 

72.19942 

7.79 

<.ooor 

-33.33182 

2.715898 

-1227 

<.ooor 

-662.512 

27.15714 

-2440 

<.ooor 

-0.005165 

0.001003 

-5.15 

<.ooor 

-0.120308 

0.010921 

-11.02 

<.ooor 

-2866.773 

673.4763 

-4.26 

<.ooor 

135.38315 

24.1139 

5.61 

<.ooor 

2.3921e-6 

4.485e-7 

5.33 

<.ooor 

1980.2703 

278.714 

7.11 

<.ooor 


The parameter estimates shown in Figure 83, formulate the prediction 
model shown in Figure 84. 


Figure 84. Test Case 2, Prediction model for Total Cost of Recruiting 

! Prediction Expression 

-126.5044997643 
+ 0.02668971455148 * NCO 
+ 562.697816223402 * TSC I-mA 
+ -33.3318158052 * Unemployment 
+ -662.51203008942‘Relative Pay 

+ ( NCO - 35000.0620155039) * ((Unemployment - 6.00310077519381 * -0.0051646636056) 
+ (NCO-35000.0620155039)*((RelativePay -l)*-0.1203081976383) 

+ (TSCI-mA-0.77503875968992)* ((RelativePay -l)*-2866.7729142712] 

+ (Unemployment - 6.0031007751938) * ((Relative Pay -1) * 135.383146993767) 

+ (NCO - 35000.0620155039) * ((NCO - 35000.0620155039) * 0.00000239213948) 

+ (Relative Pay -1) * ((Relative Pay -1) * 1980.27034394266) 
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As in Test Case 1, this prediction model indicates the presence of variable 
interactions and non-linear effects. Test Case 2 appears to be highly influenced 
by multi-variable interactions. Where Test Case 1 had just one multi-variable 
interaction and two quadratic terms, Test Case 2’s prediction model has four 
multi-variable interactions and two quadratic terms. The regression model shown 
in Figure 84 provides evidence to believe that the addition of these two policy 
uncertainties (i.e., percentage of high quality recruits and decrease in QMA) does 
increase the complexity of recruiting resource allocation and effects the total cost 
of recruiting. 

The summary of fit for Test Case 2’s prediction model is shown in Figure 
85. 


Figure 85. Summary of Fit for Test Case 2’s Prediction Model for Total 

Cost of Recruiting 


Summary of Fit 

RSquare 

0.935761 

RSquare Adj 

0.93031S 

Root Mean Square Error 

35.S9242 

Mean of Re&pon&e 

42S,367S 

Observations (or Sum Wgts) 

129 


This prediction model explains over 93 percent of the variance in the total 
cost of recruiting. 

The actual versus predicted plot in Figure 86 illustrates how the prediction 
model compares to the actual data for FY 2017. 
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Figure 86. Test Case 2, Actual versus Predicted Plot 


I Actual by Predicted Plot 



300 400 500 600 700 800 900 1000 
Total Cost of Recaiiting Predicted P<.0001 
RSq=0.94 RMSE=35.892 


The factors that were determined to be influential through the partition tree 
and stepwise regression are fit in scatterplot matrices to visualize trends or 
relationships of the data, as shown in Figure 87. QMA was also included for 
comparison even though it is not considered an influential factor. 


Figure 87. Test Case 2: Scatterplot Matrices of Influential Factors 


* Scatterplot Matrix 
1000 


II 




••‘•‘s’ *••• 

• •• < ,%••• ... ^ y* 

” iiliMlp'si'is’ 


30000 33000 35000 37000 39000 
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i 45 5 55 « 65 7 7i 6 05 


1 10511 
Watrr* 

Pay 


0.76 05052 

TVIBA 


1554000 1557000 1560000 
QMA 


As in Test Case 1, relative pay and NCO follow similar trends. Both 
scatterplot graphs for the percentage of high quality recruits show a slightly 
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upward linear trend. This indicates that an increased percentage of high quality 
recruits requires more resourcing to advertising, thus resulting in high overall 
recruiting costs. Both unemployment rate graphs show a slight downward trend, 
indicating that the total cost of recruiting and the total cost of advertising 
decreases as the unemployment rate increases. The recruit quality scatterplots 
suggest that as the requirement for recruit quality increases, more funds need to 
be allocated to advertising and the total cost of recruiting increases. Both 
scatterplots for QMA do not indicate any discernible trends. 

D. FULL FACTORIAL COMPARISON 

The NOLH DOE technique is the foundation for PROM-WED’s data 
farming wrapper. Coupled with PROM-WED’s GUI, users are able to design, 
populate, and execute space-filling experimental designs quickly and easily. 
Without the NOLH DOE, PROM-WED’s data farming wrapper would not be as 
effective. 

As previously described in Chapter II, the NOLH DOE method is an 
alternative to the straightforward full factorial method. A modified version of Test 
Case 1 is used to demonstrate what a potential full factorial could look like. This 
design tests three variables at only nine levels each. Table 16 shows an 
illustrative example of what nine levels for each variable could look like. 


Table 16. Full Factorial Levels for Modified Test Case 1 


Levels 

Relative Pay 

Unemployment Rate 

Recruiters 

1 

0.80 

4.0% 

30,000 

2 

0.85 

4.5% 

31,000 

3 

0.90 

5.0% 

32,000 

4 

0.95 

5.5% 

33,000 

5 

1.00 

6.0% 

34,000 

6 

1.05 

6.5% 

35,000 

7 

1.10 

7.0% 

36,000 

8 

1.15 

7.5% 

37,000 

9 

1.20 

8.0% 

38,000 
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In comparison to the NOLH DOE, where each variable is tested at either 
33 or 129 levels, for this full factorial example each variable is tested over only 
nine levels. To test all possible variable interactions the full factorial DOE would 
have to be run over 729 input combinations for each FY. 729 runs for each FY 
results in 5,103 runs for all seven FY’s. This is in comparison to 231 runs for the 
33-point NOLFI design, or 903 runs for the 129-point NOLH design, which 
account for all runs over all seven FYs. The pairwise scatterplot matrices of a 
multi-level full factorial design in comparison to the 129 design point NOLH are 
shown in Figure 88. 


Figure 88. Pairwise Plots of Full Factorial versus NOLH 129 Point 

Designs 



Recruiters Pay Rate Recruiters Pay Rate 

Left: Full Factorial. Right: 128-polnt NOLH DOE. 


As is evident by these pairwise plots, the NOLH DOE is able to execute 
space-filling designs with a fraction of runs. 

Not only is the NOLH DOE method an efficient and effective alternative to 
the factorial DOE method, PROM-WED demonstrates that the NOLH DOE can 
be embedded into a model to add a robust data farming capability. The NOLH 
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DOE algorithm built in Microsoft Excel by the SEED Center for Data Farming at 
NFS provides this capability. Statistical software packages, like JMP, have a 
factorial DOE capability. However, to use this method an analyst would have to 
build the factorial DOE in JMP and import the design into Microsoft Excel. 
Embedding the NOLH DOE within the legacy PRO model alleviates this extra 
step, while also providing analysts with enhanced analytic abilities through 
efficient and effective space-filling designs that provide opportunities for robust 
sensitivity and risk analysis. 

E. DISCUSSION 

PROM-WED is an enhanced analytic tool capable of providing PRO model 
users with insights to better inform recruiting resource allocation decisions. The 
legacy PRO model produces a point-solution output, as shown in Figure 89. 


Figure 89. Legacy PRO Model Output 


Resource Run 

2015 

2016 

2017 

2018 

2019 

2020 

2021 

NCO 

35.025 

36.425 

36.800 

35.800 

35.225 

34.650 

34.650 

Capacity 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Unemployment (%) 

4,0 

4.0 

4.0 

4.0 

4.0 

4.0 

4.0 

Total Recruiters 

3.913 

3.685 

3.685 

3.685 

3.685 

3.685 

3.685 

Total Recruiter Cost (SM) 

S320.4S8 

$305,122 

$309,699 

$314 344 

$319,059 

$324616 

$330,274 

Actvertismg ($M) 

$102,921 

$264,167 

$261,119 

$184724 

$142 227 

$115,543 

$113,903 

Enlistment Bonus (SM) 

$40,971 

$36 580 

$41 340 

$40 650 

$42230 

$42 060 

$42810 

Education Incentives ($M) 

$0,000 

$0,000 

$0,000 

$0,000 

$0 000 

$0,000 

$0,000 

LRP ($M) 

$7,440 

$11,220 

$11,280 

$11,380 

$11,430 

$11 460 

$11,670 

HSDG 

95% 

95% 

95% 

95% 

95% 

95% 

95% 

TSC l-IIIA 

70% 

70% 

70% 

70% 

70% 

70% 

70% 

Total Cost ($M) 

$471,820 

$617,089 

$623 438 

$551,098 

$514,946 

$493681 

$498,656 


POM FY17 version of the PRO model. 


As showcased in this chapter, PROM-WED provides users with the ablility 
to efficiently and effectively grow space-filling designs that produce data sets of 
33 or 129 points in minutes. This means that 33 or 129 data points as shown in 
Figure 89 are produced by only one run of PROM-WED. PROM-WED not only 
grows data, it also facilitates basic statistical analysis and allows for further 
exploration using a statistical software package to better inform decision makers 
on the optimal allocation of hundreds of millions of dollars to advertisements, 
enlistment bonuses, and recommended number of Navy recruiters in the field. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


Through design of experiment techniques, PROM-WED provides PRO 
model users with an enhanced analytic tool capable of producing valuable 
insights into the optimal allocation of recruiting resources. Based on the findings 
of this study, each research question presented in Chapter I is answered. 
Recommendations for further work are also presented. 

A. RESEARCH QUESTION 1 

How can design of experiment techniques better inform decision maker’s 
determination of the optimal and robust combination of recruiting resources? 

Efficient DOE techniques help better inform decision makers on the 
optimal allocation of recruiting resources through the efficient and effective 
implementation of space-filling designs. Embedding the PRO model into a data 
farming environment provides users with the ability to execute space-filling 
design of experiments. Through a single PROM-WED excursion, it is possible to 
test 33 or 129 legacy PRO model scenarios. Each excursion is able to test how 
uncertainties and variations in controllable and uncontrollable factors may affect 
the allocation of recruiting resources. In this study. Test Case 1 and Test Case 2 
are proof-of-concept examples. As demonstrated through Test Case 1, the most 
expensive resource is the number of recruiters in the field. However, it is 
apparent that the total cost of recruiting is highly dependent upon the allocation 
of funds to advertising. In order from high to low influence: the new accession 
mission, relative pay, and unemployment rate drive the amount of resources 
allocated to advertising. As for the additional policy factors included in the 
legalization of marijuana scenario explored in Test Case 2, there is evidence to 
believe that increasing the percentage of high quality recruits has a greater effect 
on the total cost of recruiting than the decrease in QMA. These few examples 
show only a small spectrum of the vast amount of information that PROM-WED 
can provide. Therefore, by using DOE techniques, PROM-WED is able to grow 
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PRO model data in a systematic and controlled way. By controlling variable 
uncertainties and interactions, analysts are able to gain insights such as the ones 
just described. These insights help better inform decision makers on determining 
the optimal and robust allocation of recruiting resources. 

B. RESEARCH QUESTION 2 

How can efficient design of experiment techniques be incorporated into the PRO 
model for future, on-the-spot risk, and sensitivity analysis? 

The PRO model is embedded into a data farming environment through the 
implementation of the Microsoft Excel NOLH DOE algorithm made available by 
the SEED Center for Data Farming. An enhanced GUI allows users to populate 
the NOLH DOE worksheet for each factor they would like to vary. The NOLH 
DOE algorithm automatically generates values for either 33 or 129 levels for 
each variable. Code is written to loop over each combination of 33 or 129 
different scenarios. The result is a data set of 33 or 129 PRO model runs for 
each PROM-WED excursion. PROM-WED provides automatically generated 
analysis in Microsoft Excel for on-the-spot risk and sensitivity analysis. To take 
advantage of the space-filling qualities that the NOLH DOE provides, results from 
using the 129-point design can be explored using any available software 
package, like JMP. 

C. RESEARCH QUESTION 3 

Can an enhanced PRO model give decision-makers a robust solution for the 
optimal allocation of recruiting resources? 

An enhanced PRO model allows analysts to understand how uncertainties 
and fluctuations in controllable and uncontrollable factors affect the allocation of 
recruiting resources. A robust solution can be interpreted through two lenses: (1) 
resiliency, or (2) gained insight. A robust solution for the optimal allocation of 
recruiting resources in terms of resiliency is one that is not overly affected by 
variations in uncontrollable factors, to include economic uncertainties such as 
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unemployment rates, or controllable factors, such as increasing the percentage 
of high quality recruits. Test Case 1 provides insights to decision makers 
regarding the optimal allocation of recruiting resources that is impervious to best 
case, worst case, and most likely economic conditions. For example, comparing 
the program of record and PROM-WED’s allocation of recruiting resources for 
Test Case 1, there is evidence to believe that the pre-determined recruiting 
allocation budget was within the same range of spending as PROM-WED’s 
solution. 

An alternative approach to interpreting robustness is through assessing 
the value of information gained through the data. PROM-WED provides analysts 
with the capability to data farm the PRO model. Using data farming, PROM-WED 
grows PRO model data in an efficient and space-filling way. Improved 
understanding of the solution space can range from basic sensitivity and risk 
analysis of the decision variables presented in PROM-WED’s automatically 
generated decision support capability, to gaining insights into how uncertainties 
in input factors affect the optimal allocation of recruiting resources using a 
software package like JMP. Valuable insights like these help analysts better 
inform decision-makers on how factors such as uncertain unemployment rates, a 
proposed policy change, or constrained resources can affect the optimal 
allocation of recruiting resources. 

D. FUTURE WORK 

The focus of this research was to enhance the existing PRO model with 
an efficient design of experiments capability. PROM-WED successfully data 
farms the PRO model’s traditional run option. Recommendations for further work 
are separated into three sections. The first section addresses additional ways to 
improve PROM-WED. The second section addresses the opportunity to study 
and improve the PRO model’s underlying mathematical construct. The last 
section addresses the opportunity to enhance any Microsoft Excel based model 
with techniques or methods employed in this research. 
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1. Capacity Run Capability and Additional Design Options 

Further work is recommended to enhance PROM-WED with the addition 
of the capacity run option along with more design of experiment choices. While 
the capacity run option was briefly explored as a part of this research, additional 
work needs to be done to ensure that the data farming wrapper correctly enters 
input values in the appropriate locations within the PRO model’s simulation 
worksheets, and extracts the correct output data. Once the data farming wrapper 
for the capacity run option is complete, its automatically generated decision 
support capability can be refined. Figure 90 shows a graph that a senior analyst 
at N1 requested to be included in the capacity run’s automatically generated 
decision support analysis. 


Figure 90. Example Capacity Run Graph 



The new accession mission is shown in red and the expected capacity 
with a 95 percent confidence interval is shown in blue. This graph illustrates 
where the Navy has either budgeted an excess or deficient amount of resources 
to meet the recruiting mission. For example, in FYs 2020 and 2021, the Navy can 
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expect to recruit approximately 36,500 new recruits each year when only 
approximately 34,650 are needed. Since the NCO missions for FYs 2016, 2017, 
and 2018 are within the 95 percent confidence interval, there is evidence to 
believe that the pre-determined allocation of recruiting resources will be sufficient 
for those FYs. 

Along with fully integrating the capacity run option into PROM-WED, work 
can be done to add other designs to PROM-WED’s data farming wrapper. This 
will allow analysts to explore a broader realm of possibilities to gain additional 
insights about the complex solution space. 

2. Recruiting Cost Function 

For the purpose of this research, it was assumed that the PRO model 
accurately models active duty enlisted recruiting resource allocation. If this 
assumption were relaxed, the following additional research is suggested. 

Within the “black box” of the recruiting cost function, elasticities can act as 
another variable with uncertainties. Currently, the elasticities are updated 
annually based on actual data from the previous FY. Therefore, further work can 
be done to include elasticities within PROM-WED’s data farming wrapper. Also, 
future work can be done to explore the relevancy of the recruiting cost function in 
current recruiting practice. 

The Navy is interested in incorporating the active duty officer, reserve 
officer, and reserve enlisted recruiting missions into the PRO model. This is a 
unique challenge since there are many diverse and unique communities within 
the active duty officer corps alone that require targeted recruiting initiatives. For 
example. Navy Doctors are often incentivized to join the Navy through a loan 
repayment program that alleviates medical school debt, or signing bonuses. On 
the other hand, loan repayment programs and signing bonuses are not available 
to prospective general line officers. Consequently, to recruit general line officers, 
large amounts of recruiting resources may be allocated to advertising in order to 
pay for college career fair booths. 
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Therefore, future work can be done to adapt the recruiting cost function to 
model the attributes of each unique recruiting mission. This additional work will 
provide analysts with an enhanced model that can help decision makers 
determine the optimal allocation of recruiting resources for the full spectrum of 
Navy recruiting. 

3. Apply Data Farming to Another Model! 

The methodology used to develop PROM-WED can be applied to any 
model built in Microsoft Excel. The NOLH DOE algorithms can be embedded into 
any Microsoft Excel model. Code similar to what is found in Appendix B can be 
written to loop over each design point of the NOLH. The resulting product is an 
enhanced tool that provides an efficient way to construct, run, and analyze a 
model using space-filling experimental designs. 
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APPENDIX A. 129-POINT NOLH DOE WORKSHEET 


2 low level 

0.0001 

0.04 

3913 

7.44 

34.8264 40.971 

35025 

0.7 

0.95 

0.4 1883304 

3 high level 

0.0001 

0.04 

3913 

7.44 

34.8264 40.971 

35025 

0.7 

0.95 

0.4 1883304 

4 decimals 

4 

2 

0 

3 

3 3 

0 

2 

2 

6 0 

5 :tor name 

NCF + College First Unemployment Rate 

Recruiters 

LRP 

Advertising (AC Enl. Only) EB 

NCO (50% BoY DEP) TSC l-IIIA HSDG 

Relative Pay QMA 

6 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

04 1883304 

7 2017 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

8 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

9 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

10 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

11 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

12 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

13 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

04 1883304 

14 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

15 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

16 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

17 

0.0001 

0.04 

3913 

7.44 

34.826 40,971 

35025 

0.7 

0.95 

0.4 1883304 

18 I I 

1 0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

19 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

20 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

04 1883304 

21 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

22 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

23 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

24 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

25 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

26 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

0.4 1883304 

27 

0.0001 

0.04 

3913 

7.44 

34.826 40.971 

35025 

0.7 

0.95 

04 1883304 

28 

0.0001 
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APPENDIX B. DATA FARMING SUBROUTINE 


Option Explicit 
Sub NOLH33loop() 

‘FY Loop 

Dim wsNames As Variant 
Dim wsCurrent As Variant 
Dim I As Long 
Dim j As Long 

wsNames = Array(“Sheet6,” “Sheetl 0,” “Sheetl 1“Sheetl 2,” “Sheetl 3,” 
“Sheet14,” “Sheetl 5”) 

For Each wsCurrent In wsNames 

‘With Worksheets(wsCurrent) 

If wsCurrent = “Sheet6” Then Call NOLH33input15 
If wsCurrent = “Sheetl 0” Then Call NOLH33input16 
If wsCurrent = “Sheetl 1 ” Then Call NOLH33input17 
If wsCurrent = “Sheetl 2” Then Call NOLFI33input18 
If wsCurrent = “Sheetl 3” Then Call NOLH33input19 
If wsCurrent = “Sheetl 4” Then Call NOLFI33input20 
If wsCurrent = “Sheetl 5” Then Call NOLH33input21 
Next wsCurrent 

End Sub 


Sub NOLFI33input15() 

Dim iterationNum As Long 

‘Update Model year on Sim Tab 
Sheet9.Range(“B3”) = Sheets. Range(“A7”) 

For iterationNum = 1 To 33 

‘Ed Benefits 

Sheets.Range(“D17”) = Sheets.Range(“B” & 4 + iterationNum) 

‘UE Rates 

Sheets.Range(“C50”) = 100 * Sheets.Range(“C” & 4 + iterationNum) 
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‘Recruiters 

Sheets.Range(“D19”) = Sheets.Range(“D” & 4 + iterationNum) 

‘LRP 

Sheets.Range(“D20”) = Sheets.Range(“E” & 4 + iterationNum) 

‘Advertising 

Sheets.Range(“D21”) = Sheets.Range(“F” & 4 + iterationNum) 

‘EB 

Sheets.Range(“D22”) = Sheets.Range(“G” & 4 + iterationNum) 

‘NCO 

Sheets.Range(“E11”) = Sheets.Range(“H” & 4 + iterationNum) 

‘Sheets.Range(“D23”) = Sheets.Range(“H” & 4 + iterationNum) 

‘Sheets.Range(“D12”) = Sheets.Range(“H” & 4 + iterationNum) 

‘TSC l-IIIA 

Sheets.Range(“N17”) = Sheets.Range(“r’ & 4 + iterationNum) 

‘HSDG 

Sheets.Range(“N1S”) = Sheets.Range(“J” & 4 + iterationNum) 

‘Relative Pay 

Sheets.Range(“D24”) = Sheets.Range(“K” & 4 + iterationNum) 

‘QMA 

Sheets.Range(“D2S”) = Sheets.Range(“L” & 4 + iterationNum) 

‘ ‘Update Model year on Sim Tab 
‘ Sheets.Range(“B3”) = Sheets.Range(“A7”) 

Call RunTraditionalS 

Sheet24.Range(“B” & 1 + iterationNum) = Sheets.Range(“D3”) ‘NCO -> output 
‘Sheet24.Range(“B” & 1 + iterationNum) = Sheets.Range(“D4”) ‘NCO cap -> 
output 

Sheet24.Range(“C” & 1 + iterationNum) = Sheets.Range(“DS”) ‘Unemployment - 
> output 

Sheet24.Range(“D” & 1 + iterationNum) = Sheets.Range(“DS”) ‘total recruiters -> 
output 

Sheet24.Range(“E” & 1 + iterationNum) = Sheets.Range(“D7”) ‘total recruiters 
cost to output 

Sheet24.Range(“F” & 1 + iterationNum) = Sheets.Range(“D8”) ‘advertising $ to 
output 
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Sheet24.Range(“G” & 1 + iterationNum) = Sheets.Range(“D9”) ‘EB $ to output 
Sheet24.Range(“H” & 1 + iterationNum) = Sheets.Range(“D10”) ‘ED $ to output 
to output 

Sheet24.Range(“r’ & 1 + iterationNum) = Sheets.Range(“E” & 4 + iterationNum) 
‘wsResultsFY15.Range(“r’ & 1 + design Number) = plnputWorksheet.Cells(7, 20 
+ iSimNumber) ‘LRP $ - a constant to output 

Sheet24.Range(“J” & 1 + iterationNum) = Sheets.Range(“D12”) ‘HSDG% to 
output 

Sheet24.Range(“K” & 1 + iterationNum) = Sheets.Range(“D1S”) ‘UMG% to 
output 

Sheet24.Range(“L” & 1 + iterationNum) = Sheets.Range(“K” & 4 + iterationNum) 
‘Relative Pay 

Sheet24.Range(“M” & 1 + iterationNum) = Sheets.Range(“L” & 4 + iterationNum) 
‘QMA 

Sheet24.Range(“N” & 1 + iterationNum) = Sheets.Range(“A7”) 

Next iterationNum 
End Sub 

Sub NOLHSSinput1S() 

Dim iterationNum As Long 

‘Update Model year on Sim Tab 
Sheet9.Range(“BS”) = Sheet10.Range(“A7”) 

For iterationNum = 1 To SS 

‘Ed Benefits 

Sheets.Range(“E17”) = Sheet10.Range(“B” & 4 + iterationNum) 

‘UE Rates 

Sheet9.Range(“C50”) = 100 * Sheet10.Range(“C” & 4 + iterationNum) 

‘Recruiters 

Sheets.Range(“E19”) = Sheet10.Range(“D” & 4 + iterationNum) 

‘LRP 

Sheets.Range(“D20”) = Sheet10.Range(“E” & 4 + iterationNum) 

‘Advertising 

Sheets.Range(“E21”) = Sheet10.Range(“F” & 4 + iterationNum) 

‘EB 


111 



Sheets.Range(“E22”) = Sheet10.Range(“G” & 4 + iterationNum) 

‘NCO 

Sheet9.Range(“E11”) = Sheet10.Range(“H” & 4 + iterationNum) 

‘Sheets.Range(“D23”) = Sheet10.Range(“H” & 4 + iterationNum) 

‘TSC l-IIIA 

Sheets.Range(“N17”) = Sheet10.Range(“l” & 4 + iterationNum) 

‘HSDG 

Sheets.Range(“N16”) = Sheet10.Range(“J” & 4 + iterationNum) 

‘Relative Pay 

Sheets.Range(“D24”) = Sheet10.Range(“K” & 4 + iterationNum) 

‘QMA 

Sheets.Range(“D2S”) = Sheet10.Range(“L” & 4 + iterationNum) 

Call RunTraditionalS 

Sheet24.Range(“B” & 34 + iterationNum) = Sheet3.Range(“D3”) ‘NCO -> output 
‘Sheet24.Range(“B” & 34 + iterationNum) = Sheet3.Range(“D4”) ‘NCO cap -> 
output 

Sheet24.Range(“C” & 34 + iterationNum) = Sheet3.Range(“DS”) ‘Unemployment 
-> output 

Sheet24.Range(“D” & 34 + iterationNum) = Sheet3.Range(“D6”) ‘total recruiters - 
> output 

Sheet24.Range(“E” & 34 + iterationNum) = Sheet3.Range(“D7”) ‘total recruiters 
cost to output 

Sheet24.Range(“F” & 34 + iterationNum) = Sheet3.Range(“D8”) ‘advertising $ to 
output 

Sheet24.Range(“G” & 34 + iterationNum) = Sheet3.Range(“D9”) ‘EB $ to output 
Sheet24.Range(“H” & 34 + iterationNum) = Sheet3.Range(“D10”) ‘ED $ to output 
to output 

Sheet24.Range(“r’ & 34 + iterationNum) = Sheet10.Range(“E” & 4 + 
iterationNum) 

‘wsResultsFY1S.Range(“r’ & 1 + designNumber) = plnputWorksheet.Cells(7, 20 
+ iSimNumber) ‘LRP $ - a constant to output 

Sheet24.Range(“J” & 34 + iterationNum) = Sheet3.Range(“D12”) ‘FISDG% to 
output 

Sheet24.Range(“K” & 34 + iterationNum) = Sheet3.Range(“D13”) ‘UMG% to 
output 

Sheet24.Range(“L” & 34 + iterationNum) = Sheet10.Range(“K” & 4 + 
iterationNum) ‘Relative Pay 
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Sheet24.Range(“M” & 34 + iterationNum) = Sheet10.Range(“L” & 4 + 
iterationNum) ‘QMA 

Sheet24.Range(“N” & 34 + iterationNum) = Sheet10.Range(“A7”) 
Next iterationNum 
End Sub 


Sub NOLH33input17() 

Dim iterationNum As Long 

‘Update Model year on Sim Tab 
Sheets. Range(“B3”) = Sheetl 1 .Range(“A7”) 

For iterationNum = 1 To 33 

‘Ed Benefits 

Sheets.Range(“F17”) = Sheetl 1 .Range(“B” & 4 + iterationNum) 

‘UE Rates 

Sheets.Range(“C50”) = 100 * Sheetll.Range(“C” & 4 + iterationNum) 
‘Recruiters 

Sheets.Range(“F1 S”) = Sheetl 1 .Range(“D” & 4 + iterationNum) 

‘LRP 

Sheets.Range(“D20”) = Sheetl 1 .Range(“E” & 4 + iterationNum) 
‘Advertising 

Sheets.Range(“F21”) = Sheetl 1 .Range(“F” & 4 + iterationNum) 

‘EB 

Sheets.Range(“F22”) = Sheetl 1 .Range(“G” & 4 + iterationNum) 

‘NCO 

Sheets.Range(“E11”) = Sheetl 1 .Range(“FI” & 4 + iterationNum) 
‘Sheets.Range(“D23”) = Sheetl 1 .Range(“H” & 4 + iterationNum) 

‘TSC l-IIIA 

Sheets.Range(“N17”) = Sheetl 1 .Range(“r’ & 4 + iterationNum) 


‘HSDG 

Sheets.Range(“N16”) = Sheetl 1 .Range(“J” & 4 + iterationNum) 
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‘Relative Pay 

Sheets.Range(“D24”) = Sheetl 1 .Range(“K” & 4 + iterationNum) 

‘QMA 

Sheets.Range(“D25”) = Sheetl 1 .Range(“L” & 4 + iterationNum) 

Call RunTraditionaie 

Sheet24.Range(“B” & 67 + iterationNum) = Sheets.Range(“D3”) ‘NCO -> output 
Sheet24.Range(“C” & 67 + iterationNum) = Sheets.Range(“D6”) ‘Unemployment 
-> output 

Sheet24.Range(“D” & 67 + iterationNum) = Sheets.Range(“D6”) ‘total recruiters - 
> output 

Sheet24.Range(“E” & 67 + iterationNum) = Sheets.Range(“D7”) ‘total recruiters 
cost to output 

Sheet24.Range(“F” & 67 + iterationNum) = Sheets.Range(“D8”) ‘advertising $ to 
output 

Sheet24.Range(“G” & 67 + iterationNum) = Sheets.Range(“D9”) ‘EB $ to output 
Sheet24.Range(“H” & 67 + iterationNum) = Sheets.Range(“D10”) ‘ED $ to output 
to output 

Sheet24.Range(“r’ & 67 + iterationNum) = Sheetl 1 .Range(“E” & 4 + 
iterationNum) 

‘wsResultsFY16.Range(“r’ & 1 + designNumber) = plnputWorksheet.Cells(7, 20 
+ iSimNumber) ‘LRP $ - a constant to output 

Sheet24.Range(“J” & 67 + iterationNum) = Sheets.Range(“D12”) ‘FISDG% to 
output 

Sheet24.Range(“K” & 67 + iterationNum) = Sheets.Range(“D13”) ‘UMG% to 
output 

Sheet24.Range(“L” & 67 + iterationNum) = Sheetl 1 .Range(“K” & 4 + 
iterationNum) ‘Relative Pay 

Sheet24.Range(“M” & 67 + iterationNum) = Sheetl 1 .Range(“L” & 4 + 
iterationNum) ‘QMA 

Sheet24.Range(“N” & 67 + iterationNum) = Sheetl 1 .Range(“A7”) 

Next iterationNum 
End Sub 

Sub NOLFI33input18() 

Dim iterationNum As Long 

‘Update Model year on Sim Tab 
Sheet9.Range(“B3”) = Sheetl 2. Range(“A7”) 
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For iterationNum = 1 To 33 


‘Ed Benefits 

Sheets.Range(“G17”) = Sheet12.Range(“B” & 4 + iterationNum) 

‘UE Rates 

Sheets.Range(“C50”) = 100 * Sheet12.Range(“C” & 4 + iterationNum) 
‘Recruiters 

Sheets.Range(“G 19”) = Sheet12.Range(‘‘D” & 4 + iterationNum) 

‘LRP 

Sheets.Range(‘‘D20”) = Sheet12.Range(‘‘E” & 4 + iterationNum) 
‘Advertising 

Sheets.Range(‘‘G21”) = Sheet12.Range(‘‘F” & 4 + iterationNum) 

‘EB 

Sheets.Range(‘‘G22”) = Sheet12.Range(‘‘G” & 4 + iterationNum) 

‘NCO 

Sheets.Range(‘‘E11”) = Sheet12.Range(‘‘FI” & 4 + iterationNum) 
‘Sheets.Range(‘‘D23”) = Sheet12.Range(‘‘FI” & 4 + iterationNum) 


‘TSC l-IIIA 

Sheets.Range(‘‘N 17”) = Sheet12.Range(‘‘l” & 4 + iterationNum) 

‘HSDG 

Sheets.Range(‘‘N 16”) = Sheet12.Range(‘‘J” & 4 + iterationNum) 

‘Relative Pay 

Sheets.Range(‘‘D24”) = Sheet12.Range(‘‘K” & 4 + iterationNum) 

‘QMA 

Sheets.Range(‘‘D2S”) = Sheet12.Range(‘‘L” & 4 + iterationNum) 

Call RunTraditionalS 

Sheet24.Range(‘‘B” & 100 + iterationNum) = Sheets.Range(‘‘D3”) ‘NCO -> output 
Sheet24.Range(‘‘C” & 100 + iterationNum) = Sheets.Range(‘‘DS”) 

‘Unemployment -> output 

Sheet24.Range(‘‘D” & 100 + iterationNum) = Sheets.Range(‘‘D6”) ‘total recruiters 
-> output 

Sheet24.Range(‘‘E” & 100 + iterationNum) = Sheets.Range(‘‘D7”) ‘total recruiters 
cost to output 
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Sheet24.Range(“F” & 100 + iterationNum) = Sheets.Range(“D8”) ‘advertising $ 
to output 

Sheet24.Range(“G” & 100 + iterationNum) = Sheets.Range(“D9”) ‘EB $ to output 
Sheet24.Range(“H” & 100 + iterationNum) = Sheets.Range(“D10”) ‘ED $ to 
output to output 

Sheet24.Range(“r’ & 100 + iterationNum) = Sheet12.Range(“E” & 4 + 
iterationNum) 

‘wsResultsFY15.Range(“r’ & 1 + designNumber) = plnputWorksheet.Cells(7, 20 
+ iSimNumber) ‘LRP $ - a constant to output 

Sheet24.Range(“J” & 100 + iterationNum) = Sheets.Range(“D12”) ‘FISDG% to 
output 

Sheet24.Range(“K” & 100 + iterationNum) = Sheets.Range(“D1S”) ‘UMG% to 
output 

Sheet24.Range(“L” & 100 + iterationNum) = Sheet12.Range(“K” & 4 + 
iterationNum) ‘Relative Pay 

Sheet24.Range(“M” & 100 + iterationNum) = Sheet12.Range(“L” & 4 + 
iterationNum) ‘QMA 

Sheet24.Range(“N” & 100 + iterationNum) = Sheet12.Range(“A7”) 

Next iterationNum 
End Sub 

Sub NOLHSSinput19() 

Dim iterationNum As Long 

‘Update Model year on Sim Tab 
Sheet9.Range(“BS”) = Sheet1S.Range(“A7”) 

For iterationNum = 1 To SS 

‘Ed Benefits 

Sheets.Range(“F117”) = Sheet1S.Range(‘‘B” & 4 + iterationNum) 

‘UE Rates 

Sheet9.Range(‘‘C50”) = 100 * Sheet1S.Range(‘‘C” & 4 + iterationNum) 

‘Recruiters 

Sheets.Range(‘‘F119”) = Sheet1S.Range(‘‘D” & 4 + iterationNum) 

‘LRP 

Sheets.Range(‘‘D20”) = Sheet1S.Range(‘‘E” & 4 + iterationNum) 

‘Advertising 
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Sheets.Range(“H21”) = Sheet13.Range(“F” & 4 + iterationNum) 

‘EB 

Sheets.Range(“H22”) = Sheet13.Range(“G” & 4 + iterationNum) 

‘NCO 

Sheets.Range(“E11”) = Sheet13.Range(“H” & 4 + iterationNum) 

‘Sheets.Range(“D23”) = Sheet13.Range(“H” & 4 + iterationNum) 

‘TSC l-IIIA 

Sheets.Range(“N17”) = Sheet13.Range(“r’ & 4 + iterationNum) 

‘HSDG 

Sheets.Range(“N16”) = Sheet13.Range(“J” & 4 + iterationNum) 

‘Relative Pay 

Sheets.Range(“D24”) = Sheet13.Range(“K” & 4 + iterationNum) 

‘QMA 

Sheets.Range(“D2S”) = Sheet13.Range(“L” & 4 + iterationNum) 

Call RunTraditionaie 

Sheet24.Range(“B” & 133 + iterationNum) = Sheet3.Range(“D3”) ‘NCO -> output 
Sheet24.Range(“C” & 133 + iterationNum) = Sheet3.Range(“DS”) 

‘Unemployment -> output 

Sheet24.Range(“D” & 133 + iterationNum) = Sheet3.Range(“D6”) ‘total recruiters 
-> output 

Sheet24.Range(“E” & 133 + iterationNum) = Sheet3.Range(“D7”) ‘total recruiters 
cost to output 

Sheet24.Range(“F” & 133 + iterationNum) = Sheet3.Range(“D8”) ‘advertising $ 
to output 

Sheet24.Range(“G” & 133 + iterationNum) = Sheet3.Range(“D9”) ‘EB $ to output 
Sheet24.Range(“FI” & 133 + iterationNum) = Sheet3.Range(“D10”) ‘ED $ to 
output to output 

Sheet24.Range(“r’ & 133 + iterationNum) = Sheet13.Range(“E” & 4 + 
iterationNum) 

‘wsResultsFY1S.Range(“r’ & 1 + designNumber) = plnputWorksheet.Cells(7, 20 
+ iSimNumber) ‘LRP $ - a constant to output 

Sheet24.Range(“J” & 133 + iterationNum) = Sheet3.Range(“D12”) ‘FISDG% to 
output 

Sheet24.Range(“K” & 133 + iterationNum) = Sheet3.Range(“D13”) ‘UMG% to 
output 

Sheet24.Range(“L” & 133 + iterationNum) = Sheet13.Range(“K” & 4 + 
iterationNum) ‘Relative Pay 
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Sheet24.Range(“M” & 133 + iterationNum) = Sheet13.Range(“L” & 4 + 
iterationNum) ‘QMA 

Sheet24.Range(“N” & 133 + iterationNum) = Sheet13.Range(“A7”) 
Next iterationNum 
End Sub 

Sub NOLH33input20() 

Dim iterationNum As Long 

‘Update Model year on Sim Tab 
Sheets. Range(“B3”) = Sheet14.Range(“A7”) 

For iterationNum = 1 To 33 

‘Ed Benefits 

Sheets.Range(“l17”) = Sheet14.Range(“B” & 4 + iterationNum) 

‘UE Rates 

Sheets.Range(“C50”) = 100 * Sheet14.Range(“C” & 4 + iterationNum) 
‘Recruiters 

Sheets.Range(“l1S”) = Sheet14.Range(“D” & 4 + iterationNum) 

‘LRP 

Sheets.Range(“D20”) = Sheet14.Range(“E” & 4 + iterationNum) 
‘Advertising 

Sheets.Range(“l21”) = Sheet14.Range(“F” & 4 + iterationNum) 

‘EB 

Sheets.Range(“l22”) = Sheet14.Range(“G” & 4 + iterationNum) 

‘NCO 

Sheets.Range(“E11”) = Sheet14.Range(“FI” & 4 + iterationNum) 
‘Sheets.Range(“D23”) = Sheet14.Range(“H” & 4 + iterationNum) 

‘TSC l-IIIA 

Sheets.Range(“N17”) = Sheet14.Range(“r’ & 4 + iterationNum) 

‘HSDG 

Sheets.Range(“N16”) = Sheet14.Range(“J” & 4 + iterationNum) 
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‘Relative Pay 

Sheets.Range(“D24”) = Sheet14.Range(“K” & 4 + iterationNum) 

‘QMA 

Sheets.Range(“D25”) = Sheet14.Range(“L” & 4 + iterationNum) 

Call RunTraditionaie 

Sheet24.Range(“B” & 166 + iterationNum) = Sheets.Range(“D3”) ‘NCO -> output 
Sheet24.Range(“C” & 166 + iterationNum) = Sheets.Range(“D6”) 

‘Unemployment -> output 

Sheet24.Range(“D” & 166 + iterationNum) = Sheets.Range(“D6”) ‘total recruiters 
-> output 

Sheet24.Range(“E” & 166 + iterationNum) = Sheets.Range(“D7”) ‘total recruiters 
cost to output 

Sheet24.Range(“F” & 166 + iterationNum) = Sheets.Range(“D8”) ‘advertising $ 
to output 

Sheet24.Range(“G” & 166 + iterationNum) = Sheets.Range(“D9”) ‘EB $ to output 
Sheet24.Range(“H” & 166 + iterationNum) = Sheets.Range(“D10”) ‘ED $ to 
output to output 

Sheet24.Range(“r’ & 166 + iterationNum) = Sheet14.Range(“E” & 4 + 
iterationNum) 

‘wsResultsFY16.Range(“r’ & 1 + designNumber) = plnputWorksheet.Cells(7, 20 
+ iSimNumber) ‘LRP $ - a constant to output 

Sheet24.Range(“J” & 166 + iterationNum) = Sheets.Range(“D12”) ‘FISDG% to 
output 

Sheet24.Range(“K” & 166 + iterationNum) = Sheets.Range(“D13”) ‘UMG% to 
output 

Sheet24.Range(“L” & 166 + iterationNum) = Sheet14.Range(“K” & 4 + 
iterationNum) ‘Relative Pay 

Sheet24.Range(“M” & 166 + iterationNum) = Sheet14.Range(“L” & 4 + 
iterationNum) ‘QMA 

Sheet24.Range(“N” & 166 + iterationNum) = Sheet14.Range(“A7”) 

Next iterationNum 
End Sub 

Sub NOLH33input21() 

Dim iterationNum As Long 

‘Update Model year on Sim Tab 
Sheet9.Range(“B3”) = Sheet15.Range(“A7”) 
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For iterationNum = 1 To 33 


‘Ed Benefits 

Sheets.Range(“J 17”) = Sheet15.Range(“B” & 4 + iterationNum) 

‘UE Rates 

Sheets.Range(“C50”) = 100 * Sheet15.Range(“C” & 4 + iterationNum) 
‘Recruiters 

Sheets.Range(‘‘J 19”) = Sheet1S.Range(‘‘D” & 4 + iterationNum) 

‘LRP 

Sheets.Range(‘‘D20”) = Sheet1S.Range(‘‘E” & 4 + iterationNum) 
‘Advertising 

Sheets.Range(‘‘J21”) = Sheet1S.Range(‘‘F” & 4 + iterationNum) 

‘EB 

Sheets.Range(‘‘J22”) = Sheet1S.Range(‘‘G” & 4 + iterationNum) 

‘NCO 

Sheets.Range(‘‘E11”) = Sheet1S.Range(‘‘FI” & 4 + iterationNum) 
‘Sheets.Range(‘‘D23”) = Sheet1S.Range(‘‘FI” & 4 + iterationNum) 


‘TSC l-IIIA 

Sheets.Range(‘‘N 17”) = Sheet1S.Range(‘‘l” & 4 + iterationNum) 

‘HSDG 

Sheets.Range(‘‘N 16”) = Sheet1S.Range(‘‘J” & 4 + iterationNum) 

‘Relative Pay 

Sheets.Range(‘‘D24”) = Sheet1S.Range(‘‘K” & 4 + iterationNum) 

‘QMA 

Sheets.Range(‘‘D2S”) = Sheet1S.Range(‘‘L” & 4 + iterationNum) 

Call RunTraditionalS 

Sheet24.Range(‘‘B” & 199 + iterationNum) = Sheets.Range(‘‘D3”) ‘NCO -> output 
Sheet24.Range(‘‘C” & 199 + iterationNum) = Sheets.Range(‘‘DS”) 

‘Unemployment -> output 

Sheet24.Range(‘‘D” & 199 + iterationNum) = Sheets.Range(‘‘D6”) ‘total recruiters 
-> output 

Sheet24.Range(‘‘E” & 199 + iterationNum) = Sheets.Range(‘‘D7”) ‘total recruiters 
cost to output 
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Sheet24.Range(“F” & 199 + iterationNum) = Sheets.Range(“D8”) ‘advertising $ 
to output 

Sheet24.Range(“G” & 199 + iterationNum) = Sheets.Range(“D9”) ‘EB $ to output 
Sheet24.Range(“H” & 199 + iterationNum) = Sheets.Range(“D10”) ‘ED $ to 
output to output 

Sheet24.Range(“r’ & 199 + iterationNum) = Sheet15.Range(“E” & 4 + 
iterationNum) 

‘wsResultsFY15.Range(“r’ & 1 + designNumber) = plnputWorksheet.Cells(7, 20 
+ iSimNumber) ‘LRP $ - a constant to output 

Sheet24.Range(“J” & 199 + iterationNum) = Sheets.Range(“D12”) ‘FISDG% to 
output 

Sheet24.Range(“K” & 199 + iterationNum) = Sheets.Range(“D1S”) ‘UMG% to 
output 

Sheet24.Range(“L” & 199 + iterationNum) = Sheet15.Range(“K” & 4 + 
iterationNum) ‘Relative Pay 

Sheet24.Range(“M” & 199 + iterationNum) = Sheet15.Range(“L” & 4 + 
iterationNum) ‘QMA 

Sheet24.Range(“N” & 199 + iterationNum) = Sheet15.Range(“A7”) 

Next iterationNum 
End Sub 

Sub RunTraditional6() 

Dim pCalcWorksheet As Worksheet 

Dim pResultWorksheet As Worksheet 

Dim pInputWorksheet As Worksheet 

Dim pUserInterfaceWorksheet As Worksheet 

Dim iSimNumber As Long 

Dim iNumSimulations As Long 

Dim iOldCalcalculationSetting As Long 

Dim pUserWorksheet As Worksheet 

Dim pTradRunsWorksheet As Worksheet 

Dim casenum As Long 

Dim designPoints As Long 

Set pCalcWorksheet = ThisWorkbook.Worksheets(“Simulation”) 

Set pResultWorksheet = ThisWorkbook.Worksheets(“Output”) 

Set pInputWorksheet = ThisWorkbook.Worksheets(“lnput”) 

Set pUserInterfaceWorksheet = ThisWorkbook.Worksheets(“User Interface”) 

Set pTradRunsWorksheet = ThisWorkbook.Worksheets(‘‘Traditional Runs”) 
iOldCalcalculationSetting = Application.Calculation 
iNumSimulations = 7 

Application.ScreenUpdating = False 


121 



Application.Calculation = xlCalculationManual 
pResultWorksheet.Columns(“B:Q”).Clear 
ThisWorkbook.Worksheets(“Simulation”).Activate 
pCalcWorksheet.Cells(8, 2) = “User Defined” 

‘pCalcWorksheet.Cells(8, 2) = “Model Year” 
pResultWorksheet.Cells(2, 3) = “Resource Run” 
pCalcWorksheet.Cells(14, 2) = pUserlnterfaceWorksheet.Cells(27, 4) 
pCalcWorksheet.Cells(15, 2) = pUserlnterfaceWorksheet.Cells(28, 4) 
pCalcWorksheet.Cells(16, 2) = pUserlnterfaceWorksheet.Cells(29, 4) 
pCalcWorksheet.Cells(17, 2) = pUserlnterfaceWorksheet.Cells(30, 4) 

For casenum = 1 To 3 ‘Run through High UE, Base UE, Low UE scenarios 
ThisWorkbook.Worksheets(“User Interface”).Activate 
pUserlnterfaceWorksheet.Cells(18, 17) = casenum 
ThisWorkbook.Worksheets(“Simulation”).Activate 


‘Just for FY 2015 
For iSimNumber = 1 To 1 

‘pCalcWorksheet.Cells(3, 2) = pCalcWorksheet.Cells(9 + iSimNumber, 29) 

‘Updates the model year 

Application.Calculate ‘Recalculates sheet 

‘pResultWorksheet.Cells(2, 3 + iSimNumber) = iSimNumber + 2014 ‘Copies 
Model Year to output 

pResultWorksheet.Cells(3, 3 + iSimNumber) = pCalcWorksheet.Cells(11,5) ‘4 
‘Copies NCO to output 

pResultWorksheet.Cells(4, 3 + iSimNumber) = “N/A” ‘Copies Capacity to output 
pResultWorksheet.Cells(5, 3 + iSimNumber) = pCalcWorksheet.Cells(50, 3) 
‘Copies unemployment to output 

pResultWorksheet.Cells(6, 3 + iSimNumber) = pCalcWorksheet.Cells(8, 9) 
‘Copies total recruiters to output 

pResultWorksheet.Cells(7, 3 + iSimNumber) = pCalcWorksheet.Cells(8, 10) 
‘Copies total recruiters cost to output 

pResultWorksheet.Cells(8, 3 + iSimNumber) = pCalcWorksheet.Cells(9, 10) 
‘Copies advertising $ to output 

pResultWorksheet.Cells(9, 3 + iSimNumber) = pCalcWorksheet.Cells(10, 10) 
‘Copies EB $ to output 

pResultWorksheet.Cells(10, 3 + iSimNumber) = pCalcWorksheet.Cells(11, 10) 
‘Copies ED $ to output to output 

pResultWorksheet.Cells(11,3 + iSimNumber) = plnputWorksheet.Cells(7, 20 + 
iSimNumber) ‘Copies LRP $ - a constant to output 

pResultWorksheet.Cells(12, 3 + iSimNumber) = pCalcWorksheet.Cells(9, 6) 
‘Copies HSDG% to output 
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pResultWorksheet.Cells(13, 3 + iSimNumber) = pCalcWorksheet.Cells(11,6) 
‘Copies UMG% to output 
‘pResultWorksheet.Cells(14, 3 + iSimNumber) = 
WorksheetFunction.Sum(pResultWorksheet.Cells(7, 3 + iSimNumber), 
pResultWorksheet.Cells(8, 3 + iSimNumber), pResultWorksheet.Cells(9, 3 + 
iSimNumber), pResultWorksheet.Cells(10, 3 + iSimNumber), 
pResultWorksheet.Cells(11,3 + iSimNumber)) 

ThisWorkbook.Worksheets(“Traditional Runs”).Activate 
pTradRunsWorksheet.Cells(7 + casenum, 1 + iSimNumber) = 
pResultWorksheet.Cells(8, 3 + iSimNumber) ‘Also enter Capacity in UE 
scenarios table 

ThisWorkbook.Worksheets(‘‘Simulation”).Activate 

Next 

Next 

ThisWorkbook.Sheets(‘‘Output”).Activate 
Polished ‘Formats output 
‘ResourceChart 

Application.Calculation = iOldCalcalculationSetting 
Application.StatusBar = False 
Application.ScreenUpdating = True 
ThisWorkbook.Sheets(‘‘Output”).Activate 
ActiveSheet.Cells(1, 1).Select 

End Sub 
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APPENDIX C. PROM-WED USER MANUAL 
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NOTICE: 

The user is cautioned that PROM-WED has not undergone formal verification 
and validation testing, and comes without any warranty. Informal testing confirms 
the outputs from PROM-WED match the output from the legacy PRO model. 


127 




I. 


WHAT IS PROM-WED 


PROM-WED embeds the legacy PRO model within a data farming 
environment. The foundation of PROM-WED’s data farming wrapper is the nearly 
orthogonal Latin hypercube (NOLH). The NOLH design of experiments (DOE) 
builds experimental designs that efficiently and effectively explore the solution 
space (Cioppa & Lucas, 2007). This good space-filling capability means that 
uncertainties and fluctuations in input variables along with multivariable 
interactions can be adequately investigated (Sanchez & Wan, 2015). 

The 33 and 129 design point NOLH designs were used to construct 
PROM-WED’s data farming wrapper. The 33-point NOLH DOE tests each 
variable at 33 levels and grows data for 33 legacy PRO model runs, whereas the 
129-point NOLH DOE tests each variable at 129 levels and grows data for 129 
legacy PRO model runs. PROM-WED’s graphical user interface (GUI) allows 
users to easily input a range of values for each input variable into the NOLH DOE 
worksheet, without need for knowledge or familiarity with data farming or DOE 
techniques (Sanchez, 2011). 

A completed PROM-WED excursion grows a data set for either 33 or 129 
data points. Automatically generated sensitivity analysis provides users with a 
basic risk assessment picture focused on the decision variables using the data 
grown by PROM-WED. Further insights into variable interactions and effects of 
input variables can be easily explored using available data analysis software. 
PROM-WED transforms the legacy PRO model into a resource that N1 can use 
to gain robust insights into the optimal allocation of recruiting resources. 
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II. OUTPUT OPTIONS 


PROM-WED provides users with decision support capabilities to analyze 
the data grown by each excursion. PROM-WED offers two decision support 
capabilities: (A) automatically generated analysis, and (B) data generated for 
further analysis requiring a statistical software package. 

A. AUTOMATICALLY GENERATED ANALYSIS 

PROM-WED’s “Decision Support Analysis” for the traditional run option 
provides users with a broad understanding of how variability in decision 
variables, controllable policy changes, and uncontrollable market factors affect 
the total cost of recruiting. This type of analysis would be appropriate for testing 
excursions during a time constrained meeting, working group, or whenever basic 
analysis needs to be generated quickly. 

An example of PROM-WED’s automatically generated analysis follows. 
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PRDM-WED 

Decision Support Analysis 




Traditional Run: 33 Design Points 
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PROM-WED 

Decision Support Analysis 



Decision Variables 


Total Cost of Recruiting 



I Advertising 
lEB 

I Recruiter Cost 


2015 2016 2017 2018 2019 2020 2021 

FY 


Total Recruiters 


Advertising 


3200 







^ 120 
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“ 2600 
o 

S 2400 

E 2200 

3 

Z 2000 
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B. JMP ANALYSIS 


Analysts will need to use a statistical software package to take full 
advantage of the data grown by PROM-WED. Therefore, data produced by 
PROM-WED is designed to be easily uploaded into a software package, such as 
JMP (JMP Pro, 2015). 

The following are examples of insights gained through analysis of PROM- 
WED data in JMP. 

1. Partition Tree 


^ Partition for Total Cost of Recruiting 



RSquare 

RMSE 

N 

Number 
of SpIKs 

AICc 

Training 

0.824 

52.07929 

129 

2 

139422 

Validation 

0.815 

53.818414 

774 




All Rows 

Count 

Mean 

Std Dev 

129 LogWorth Difference 

398.24689 57.405143 38346 

124.63855 


Advertising <139.1744 


Ad V erti si n g >=139.1744 

Count 

Mean 

Std Dev 

120 LogWorth DiffereiKe 
371.49385 78.164702 125.276 

69.85107 

Count 

Mean 

Std Dev 

9 

754.95403 

148.6531 




> Candidates 




Advertising < 5 5.0222 
Count 82 

Mean 331.82312 
Std Dev 33.803821 

Advertising >=55.0222 
Count 38 

Mean 457.09912 

Std Dev 46.655738 

> Candidates 

> Candidates 


Over 80 percent of variance in the total cost of recruiting is explained by the 
amount of funds allocated to advertising. 
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2. Stepwise Regression 

The total cost of recruiting can be formulated into a stepwise regression model: 


Prediction Expression 

150.980052850998 
+ 0.02553596931337 *NCO 
+ -24.593679044673* Unemployment 
+ -550.34801858165‘Relative Pay 

+ (NCO - 35000.0620155039) * ((Relative Pay -1] * -0.1270005027034] 

+ (NCO - 35000.0620155039 ] * ((NCO - 35000.0620155039 ] * 0.00000304992539 j 
+ (RelativePay -l]*((RelativePay -ij* 1882.37625427676] 


3. Contour Plots 


'' Contour Plot for Advertising 


40000 
39000 
38000 
37000 
36000 
NCO 35000 
34000 
33000 
32000 
31000 
30000 



0.80 0.85 0.90 0.95 1.00 1.05 1.10 1.15 1.20 
Relative Pay 


Advertising 

■ <= 50.0 
™ <= 100.0 
«« <= 150.0 
™ <= 200.0 

■ <= 250.0 

■ <= 300.0 

■ > 300.0 


The contour plot indicates that nearly half of the solution space supports a 

low advertising budget, represented by the dark blue region. The cost of 

advertising substantially increases when relative pay favors the civilian sector 

and the accession mission is high, represented by the red region. Once relative 

pay exceeds approximately 1.00, changes in the new accession mission have 

little to no effect on the amount of resources allocated to advertising. 
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STEP-BY-STEP INSTRUCTIONS TO RUN PROM-WED 


Step 1: Unzip the “PROM-WED.zip” file, and save the “PROM-WED.xIsm” file 
and “NOLH.xls” file in the same folder. This folder is where the output file 
generated by PROM-WED will be saved following the PROM-WED excursion. 


Date modified Type Size 

5/20/2016 2:16 PM Microsoft Excel 97,„ 1,136 KB 

1/20/201711:55 AM Microsoft Excel M... 2,157 KB 


m. 


PROM-'.VED ► PROM-WED 


Organize ^ Burn 

New folder 

'if Favorites 

Name 

Recently Changed 

Public 

K Desktop 

Downloads 
^ Recent Places 

gjNOLH 

Qfl PROM-WED.V2 

3 Libraries 

Documents 

S Pictures 

H Videos 


Computer 

ftoS(G) 

^ arhogartS CVXcomfoi 
My Web Sites on MS 


Network 



Step 2: Open the PROM-WED file, and ensure the “Enable Content” button is 
selected. 




Wanwif (•*«»« Hh Men «n«btc« 'he* <«i «»«r« tnaOle CeMent 



[NPsiL- . Planned Resource Optimization Model 

with Experimental Design 

(PRDM-WEO) 



Okk 10 iouncft PfiOM-WtO 


U 

FROM-VVCO 

a 

POC Lf AttkonHog^ 
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The first time you open PROM-WED, the NOLH.xls file link needs to be updated. 
To do this, select the “Edit Links...” button. 



We can't update some of the links in your workbook right now. 

You can continue without updating their values, or edit the links you think are wiong. 


Continue 


Edit links... 


To update the NOLH.xls file, click on the “Change Source...” button. 



A file search window will pop-up. Navigate to the folder where you saved the files 
after unzipping them. Select the “NOLH.xls” file, and click on the “OK” button. 

JW ^ » PfiOM-WEO -r P 
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The “Edit Links” window will pop-up. Once the “NOLH.xls” worksheet’s status 
updates to “OK,” click on the “Close” button. 


Edit Links 




Source Type Update Status 

^date Values 

NOLH.xls Worksheet A OK 

ChaQge Source... 

^en Source 

Break Link 

« ttt » 

£heck Status 

Location: C\Users\$am_gray\Desktop\PROM-WED 

Rem: 

Update: o Automatic i?l 


startup Prompt... 

Close 

_ 



Step 3: Open the PROM-WED file, and select the “PROM-WED” button to open 
the GUI. 


■ .• r Q ~ 


J^PSjL Planned Resource Optimization Model 

with Experimental Design 


CAk* Hw* ID ( mtkA miW-kVT 0 


PIKHMVtO 


POC LT Amtcmhogtrth 
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step 4: Select the appropriate starting fiscal year (FY) from the drop-down list. 
The current version of the legacy PRO model is set at a FY 2015 start. 


[^p^f Planned Resource Dptimization Model 

with Experimental Design 

(PROM-WED) 






Msrhet 


TKl-OU 

»<DC 


^ j 


fV 




MPtay >>iWI 
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step 5: To constrain a decision variable, select it from the list, and click “Fix DV” 
button. 


.-a j^NPSl/ r-. Planned Resource Dptimization Model 

with Experimental Design 

(PRDM WED) 
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OmmmI 
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1 1 1 I 
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The default data from the legacy PRO Model will automatically populate the 
“Design of Experiments Table.” 



Planned Resource Dptimization Model 

with Experimental Design 

(PROM WEO) 



twian I ^ J 


VMaUrM 




IfciM Tyy» of I 




Induce «i.^M «n#m n 


Sm Sonar* | 
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step 6: Input the range of values for the decision variable in the “Design of 
Experiments Table.” Input the low value of the range in the “Low Level” text box 
for each FY, and the high value of the range in the “High Level” text box for each 
FY. In this example, the number of recruiters is tested from 2,500 to 3,500 for 
each FY. 

Each year can be tested using different ranges. For example, to represent a 
smaller recruiter force in FY 2021, the range could be inputted as 2,000 to 2,700. 

If you want to constrain the decision variable at the default value populated by 
the legacy PRO model, select the “Fix Value” button. By selecting “Fix Value,” 
the default values for the decision variable in the “Design of Experiments Table” 
are deposited into the NOLH worksheet for each FY. This decision variable is 
now moved to the “Fixed Decision Variables” list, and the “Design of Experiments 
Table” is cleared. (If this is your course of action, continue to Step 8.) 

If you want to constrain the decision variable at one number that is different than 
the default value populated by the legacy PRO model, the same number has to 
be inputted into the “Low Level” and “High Level” text boxes. For example, if you 
want to constrain the number of recruiters in FY 2021 to 2700, then you would 
enter 2700 in both the “Low Level,” and “High Level” text boxes. 



Planned Resource Dptimization Model 

with Experimental Design 

(PROM WEO) 



»C7 • >Vit 

B 

QN* 


JiMlP 






AlMirWK OptMM 








f Sm tevnve IMnr tcwwi 

9m 
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step 7: Once the “Design of Experiments Table” is fully populated with the low 
and high levels for each FY, select the decision variable from the “Input Values” 
box, and click on the “Set Range” button. 


Planned Resource Dptimization Model 

with Experimental Design 

(PRDM-WEO) 






V«rwM»S«t 
ftrtd Dttitiom Vsfisbie* 


QMA 

TSCI-tUA 
l>wiC<oym«it lUtc 


nr 

ms 


OttiQii of CxprtlmcflU Tobir 
Bofc low Nigfi Dccanol 

Volue IrvH Level fUce« 


9913 


[smT 

fmT 

fisr 


2S00 

2S00 

2S00 

2S00 


ISDO 

3500 

»00 

3S00 


Varted Harttrt f»€ton ftrtd Hmrkrt fseton 

AftalyeM 

Select Run Type 

Option* 

Dewpn of (xperPnenl* 

TradQondfttf) 

CepiOtyRu) 

330cpgnPon(s 

U9 Ocpgrt PoeHi 


P Indude ouQM for «ntlvM n 


"»* fleer 


FY70 

rrai 




P SoveSowwte Name SceiMrte 


Run 

NOlHRun 


S>ug»<w»eMa 
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By selecting “Set Range,” the low and high values entered for this decision 
variable in the “Design of Experiments Table” are deposited into the NOLH 
worksheet for each FY. This decision variable is now moved to the “Fixed 
Decision Variables” list, and the “Design of Experiments Table” is cleared. 


rtopci 



Planned Resnurce Dptimizatinn Model 

with Experimental Design 

(PRDM-WED) 



Start nFY | FV20lS_-J 

SctVartable« 

OecnkM Vsrisbte* 


Saved Scenarios: | 

Input Values 


3 



foerdOeattoH Vsnsbte* 
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step 8: Follow Steps 5-7 to fix any other decision variables. 

Reminders: 

• If you constrain a decision variable to a number other than the default 
values populated from the legacy PRO model, as mentioned earlier enter 
the same number into the low and high level text boxes, and select the 
“Set Range” button when complete. 

• Since the PRO model solves an optimization problem, ensure that at least 
one of the following decision variables: Recruiters, Advertising or 
Enlistment Bonus (EB) remain in a “float” status. In this example, only the 
number of recruiters are fixed. 
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step 9; Once all decision variables that need to be fixed are fixed, gears shift to 
the market factors. The “Market Factors” list includes all market factors (relative 
pay, QMA and unemployment rate) and policy factors (percentage of high quality 
recruits (TSC l-IIIA), percentage of recruits with a high school diploma (HSDG), 
and NCO). Each market factor, from relative pay to NCO, must either be fixed at 
one value, or a range of values needs to be entered. 

Similar to how decision variables are fixed, select “Relative Pay” from the list of 
market factors, and select the “Add MF” button. 



Planned Resource Dptimization Model 

with Experimental Design 

(PROM-WED) 



SUrtMPr I 


Saved SceMrtes: 
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SetVanaUes 
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VanaMe Set 
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Recruters 


Verted HsHicet Fscton 

fixed Hericet fectort 


Option* 

Select Run Type 

OeMgn of CxpenmenU 

TradiMnalRu> 

33 Oeagn Rents 

CapaotyRun 

129 Des^n Ponts 


V Indude Output fer n 
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FYJO 

FY21 



n SaveScenn NameSceurto | 

Run 

NUHIlu, C«)«l 
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The default data from the legacy PRO Model automatically populates in the 
“Design of Experiments Table.” 



Planned Resnurce Optimizatinn Madel 

with Experimental Design 

(PRDM-WEO) 
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step 10; Input the range values for the market factor in the “Design of 
Experiments Table.” Input the low value of the range in the “Low Level” text box 
for each FY, and the high value of the range in the “High Level” text box for each 
FY. 

In this example, the relative pay is tested from 0.8 to 1.2 for each FY. Clicking the 
“Set Range” button deposits the low and high values entered for this market 
factor into the NOLH worksheet for each FY. 
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This market factor is now moved to the “Varied Market Factors” list, and the 
“Design of Experiments Table” is cleared. 



Planned Resource Optimization Model 

with Experimental Design 

(PRDM-WED) 



Start in rv | FYMlSj 


Saved ScefwrtOK 


3 


SetVariabln 
Deastoti Vahebhf 

FocDV 
S 

HaH»t Factors 



Adverting (AC Enl. Only} 


SalKta Tmi 
Srygk ¥t^ 




Variable Set 


FtxedDeasioo Vartabies 

Reauters 


-1 

varted Hartcet Factors 

faced Harket Factors 

RefaOve Pay 


Analyvt Optiorts 

Select Run Type Design of Experiments 

Tradtxxtal Rm 

CapaotyRixi 

33 Oe«gn Ponts 

129 Oeagn Ponts 


n Indude output for analyas r> 


r SaveScenaro Name Scenario j 

Run 


Step 11: Work through each “Market Factor” in the list, from “Relative Pay” to 
“NCO” following Steps 9-10. 

Note that each year can be tested using a different range of values for the market 
factors. For example, an annual decrease of 10,000 QMA can be entered as 
shown in the figure below. 


Design of Experiments Table 


Base 

PY Value 

Low High Decimal 

Level Level Places 

1 1883304 

1 1873304 1 1883304 | 0 


1 1883304 

1 1863304 1 1873304 | 0 


1 1883304 

1 1853304 1 1863304 | 0 

1 1883304 

1 1843304 1 1853304 | 0 


1 1883304 

1 1833304 1 1843304 | 0 

FY20 1 1883304 

1 1823304 1 1833304 | 0 

1 1883304 

1 1813304 1 1823304 | 0 


147 




































































































If you want to constrain the market factor at one number different than what is 
populated by the legacy PRO model, the same number has to be inputted into 
the “Low Level” and “High Level” text boxes. Then select the “Set Range” button. 

To constrain the market factor at the value automatically populated in the “Design 
of Experiments Table,” select the market factor from the “Input Values” box, and 
click on the “Fix Value” button. 



Planned Resource Dptimization Model 

with Experimental Design 

(PRDM-WED) 
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FY 

Base 

Value 

Lew 

Level 

High 

Level 

Decimal 

Ptaces 

FY15 

0.9S 

1 0.95 

1 0.95 

1 2 

FV16 

0.9S 

1 0.95 

1 0.95 

1 2 

nri7 

0.95 

1 0.95 

1 0.95 

1 2 

me 

0.95 

1 0.95 

1 0.95 

1 2 

me 

0.95 

t 0.95 

1 0.95 

1 2 

fY20 

0.95 

1 0.95 

1 0.95 

1 2 

FY21 

0.95 

1 0.95 

1 0.95 

1 2 
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step 12: Work through all seven market factors until they are all accounted for. A 
market factor is accounted for once it appears in either the “Varied Market 
Factors,” or “Fixed Market Factors” lists. 


y JjlNPSlL -, Planned Resource Dptimizatien Model 

with Experimental Design 

(PRDM-WED) 
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step 13: Select “Traditional Run” under “Select Run Type.” (Currently, only the 
Traditional Run option is operational). 

Automatically Generated Decision Support : 

The “33 Design Points” option is well suited for the automatically generated 
decision support analysis. The “129 Design Points” option can also be used, but 
it will take additional time to run (approximately 10 minutes versus 2-3 minutes). 
The “129 Design Points” option grows more data, resulting in a narrower 95% 
confidence interval. 

Analysis in JMP: 

The “129 Design Points” option is intended to be used for further analysis in a 
commercial statistical software package, such as JMP. 
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step 14: To save PROM-WED output to a separate .xls file for analysis in JMP, 
select the “Include output for analysis in JMP” box. This will save the PROM- 
WED output as a .xls file in the same folder that the PROM-WED model was 
saved in. 



Planned Resource Dptimization Model 

with Experimental Design 
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step 15: Once the run options are set, select the “NOLH Run” button. A 
message will pop-up providing an estimated wait time for the PROM-WED 
excursion. Click “OK.” 
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Step 16: When the PROM-WED excursion is complete, the automatically 
generated decision support analysis will appear (this is true for both the 33 and 
129 point designs). If you selected the option to output PROM-WED data for 
analysis in JMP, the .xls file named “PROMWED_Output129.xls” will appear in 
the folder that your PROM-WED model is saved in. 



Please be aware that each 129 design point output file will be named 
“PROMWED_Output129.xls.” It is recommended that you rename the file before 
running another PROM-WED excursion. 
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IV. GUIDELINES FOR ANALYSIS OF PROM-WED DATA IN JMP 


Using JMP Pro 12, the following section provides a tutorial on analysis 
techniques for PROM-WED output. Steps 1-5 explain how to upload and prepare 
the data for analysis in JMP, followed by guidance on how to conduct various 
analysis techniques. 

Analysis Techniques: 

A. Oneway Analysis of Total Cost of Recruiting by FY 

B. Explore Outliers from the Oneway Analysis Graph 

C. Select one FY to Analyze 

D. Distribution 

E. Partition Trees 

F. Stepwise Regression Model 

G. Scatterplot Matrix 
FI. Contour Plot 

Step 1 : To load the PROM-WED data into JMP, select the folder icon. 


File Tables DOE Analyze 

ILt ^ Open [ ^ ^ 

Recent Flelp 

Discovering JMP 

Beginners Tutorial 

Using JMP 




n 


Ur 
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step 2: Select the output data of interest, select the “Best Guess” option, and 
click “Open.” 



Step 3: Select the “Import” button. 
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The PROM-WED output data should appear in a table, as shown below: 


JMP_129DesPts - JMP Pro 




File Edit Tables Rows Cols DOE Analyze Graph Tools View Window Help 




•• JMP.129De5Pts > 

1 - 5 " 

Run # 

FY 

Total Cost of 
Recruiting 

Advertising 

EB 

Education 

Incentive 


1 

1 

2015 

327,1265 

35.7344 

47.1694 

0 


2 

2 

2015 

377.1404 

55.9153 

73.8082 

0 

- Columns (15/0) 

3 

3 

2015 

410.8494 

85.6254 

113.0255 

0 

^ Run 

4 

4 

2015 

283.2184 

20.127 

26.5676 

0 

jp/ 

5 

5 

2015 

359.4841 

43.8919 

57.9373 

0 

^ Total Cost of Recruiting 
^ Advertising 

JEB 

6 

6 

2015 

467,9692 

85.7103 

113.1376 

0 

7 

7 

2015 

316.5387 

18.2144 

24.043 

0 

8 

8 

2015 

464.2046 

77.1682 

101,862 

0 

^ Total Recruiters 

9 

9 

2015 

404.0421 

73.583 

97.1295 

0 

ji Recruiter Cost 

10 

10 

2015 

471.4698 

104.0235 

137.311 

0 

,JNC0 

,JLRP 

11 

11 

2015 

320.903 

30.0158 

39.6208 

0 

12 

12 

2015 

401.3847 

72.72 

95.9904 

0 

.JTSCI-IIIA 
^ Unemployment 

13 

13 

2015 

316,4949 

11.3114 

14.9311 

0 

14 

14 

2015 

566.1871 

120.0318 

158.4419 

0 

^ Relative Pay 

15 

15 

2015 

310.9625 

8.0795 

10.6649 

0 

.JQMA 

16 

16 

2015 

489,4982 

84.7874 

111.9194 

0 

Rows 

17 

17 

2015 

450.3778 

95.4969 

126.056 

0 

All rows 903 

18 

18 

2015 

422.3013 

81.1709 

107.1456 

0 

Selected 0 

19 

19 

2015 

313.7766 

33.8635 

44.6998 

0 

Excluded 0 

20 

20 

2015 

332,7022 

38.4201 

50.7146 

0 

Labelled 0 

21 

21 

2015 

382,5251 

44,468 

58.6978 

0 




Step 4: Change the FY column from “continuous” to “nominal” data, by right 
clicking on the blue triangle next to “FY,” and select “nominal” from the drop 
down menu. 


• JMP I29D«t>ts 

T 

• 

Run* 1 

m 

» Source 

1 

1 

2015 

2015 

2 

2 

• Columns (IS^) 

3 

3 

2015 

- 


4 

4 

2015 



S 

5 

2015 

• Continuous 


6 

6 

2015 



7 

7 

2015 



8 

8 

2015 

Noffunal 


9 

9 

2015 

I^RccnMierCot? 


10 

10 

2015 

JNCO 


11 

11 

2015 



12 

12 

2015 



13 

13 

2015 

Urwnployment 


14 

14 

2015 

^ Relative Pay 


15 

15 

2015 

.JQMa 


16 

16 

2015 

j • Rows 

17 

17 

2015 

Alrows 903 

18 

18 

2015 

Selected 

0 

19 

19 

2015 

Excluded 

0 

20 

20 

2015 



21 

21 

2015 
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The blue triangle next to FY will change to a red bar chart icon when JMP 
changes its classification to nominal data. 


» Columns (15/1) 

Run - _ 

4 .' _ 

^ Total Cost of Recruiting 

^ Advertising 

^EB 

^ Education Incentive 

^ Total Recruiters 

^ Recruiter Cost 

^NCO 

.JLRP 

^HSDG 

.^TSCI-mA 

^ Unemployment 

^ Relative Pay 

.JQMA 


The data is now ready to be analyzed. 
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A 


ONEWAY ANALYSIS OF TOTAL COST OF RECRUITING BY FY 


Oneway Analysis of Total Cost of Recruiting By FY 

1000 

. 

900 , • • 1 



FY 


A Quantiles 


Level 

Minimum 

10% 

25% 

Median 

75% 

90% 

Maximum 

2015 

251.8643 

283.3929 

309.1051 

349.9401 

421.2093 

521.9235 

959.1079 

2016 

258.138 

289.8711 

315.9455 

356.89 

428.4688 

529.2777 

968.0586 

2017 

261.6095 

293.5959 

320.1883 

361.2135 

433.1215 

533.85 

974.1696 

2018 

265.1709 

297.4278 

324.5511 

365.861 

437.8086 

538.5172 

980.3789 

2019 

268.7332 

301.3375 

329.0443 

370.5412 

442.3917 

543.1901 

986.5974 

2020 

272.8857 

305.6795 

334.2964 

375.5285 

447.5582 

548.4681 

993.3847 

2021 

277.2888 

310.3785 

339.8189 

380.7692 

453.0851 

554.0022 

100043 


Step 1 : To create an oneway analysis of total cost of recruiting by FY graph, 
select “Analyze” from the ribbon, and select “Fit Y by X.” 


Analyze Graph Tools View Window Help 



Distribution 



FitYbyX 



Matched Pairs 



Tabulate 


' ^ 

Fit Model 



Modeling 



Multivariate Methods 

> 


Quality and Process 

> 


Reliability and Survival 

> 


Consumer Research 

«- XL LVIJ - 

► 


Examine relationships between two 
) variables. Creates a Oneway, Bivariate, 
Contingency, or Logistic analysis 
' based on the context and modeling 
type. 

location 

centive 

0 

0 

Tot 





U 

83.2184 

20.127 

26.5676 

0 


59.4841 

43.8919 

57.9373 

0 


67.9692 

85.7103 

113.1376 

0 


16.5387 

18.2144 

24.043 

0 


64.2046 

77.1682 

101.862 

0 


04.0421 

73.583 

97.1295 

0 


71.4698 

104.0235 

137.311 

0 


320.903 

30.0158 

39.6208 

0 


01.3847 

72.72 

95.9904 

0 
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step 2: Select “Total Cost of Recruiting” from the list of columns, and select the 
“Y, Response” button. 


Ill fit Y by X * Contextual * >MP Pro 

Distribution of Y for each X. Modeling types determine analyiii. 
Select Columns 



IS Coturrms 
^Run* 


Education Incentive 
^otal Recruiters 
alRecru Iter Cost 
^NCO 


^RP 

^HSOG 

^TSCMIIA 

^Unemployment 

^RHa^e Pay 

Aqma 


Cast Selected Columns into Roles Acbon 


V, Response 

required 


Factor 

required 

Block 


Woghl 


Freq 


L...^ J 




"y. hi 

Bivanate Onewa> 

ILl b 

Logistic Contir>gency 

A AA 

I 

“Total Cost of Recruiting” should now appear in the “Y, Response” box. 


Frt Y by X - Contextual • JMP Pro 
Distnbution of Y for each X. Modelir>g types determine analysis. 
Select Columns 
• 15 Columns 
^Run 

^Advertising 

Aib 

Education Incentive 
^otal Recruiters 
^lecruiter Cost 
I ^CO 

' Arp 

Asdg 
Asci-iua 

Alnempioyment 
Relative Pay 
A^MA 


( @ 


Cast Selected Columns into Roles 


¥, Response 

^ Total Cost.. Recruiting 



I X, Factor 

required 



I Block 


I Woght I 


Freq 


By 



Acbon 

OK 

CaiKel 

Remove 

Recall 

Help 




Bivanate 

Oneway 

Ll 

Q 

Logistic 

Contingency 

j 
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step 3: Select “FY” from the list of columns, and select the “X, Factor” button. 


Hi frtV by X -Contextual-JMP Pro I 

; DistrAKJtion of Y for eoch X Modding types determine ervsfysii. 


Select Cokjnnns 
• 15 Columns 





Bivanate 

Onenay 

iL. 

D 

Logistic 

Contingency 

A 



Cast Selected Cdumns into Roles 


Y. Raxponsa 

A Total Cost.. Recruiting 

X Factor 

'Yguiied 


Block 


Weight 


Frcg 


_®Y_ 



Action 

OK 

I CarKd 


Remove 
I Racall 

H«)p 


“FY” should now appear in the “X, Factor” box. 


Fit Y by X - Contextual - JMP Pro 
Distribution of Y for each X Modefir>g types determine analysis. 
' Select Columns 
• 15 Columns 
^Run * 


iS 


^otal Cost of Recruibng 
^Advertising 

Mb 

Education Incentive 
atfTotal Recruiters 
^Recruiter Cost 

Mko 

Arp 

AfSDG 

Ascmoa 

Alnemployment 
diRelative Pay 

Ajma_ 


Cast Selected Columns into Roles 


Response 

^Total Cost.. Reenjiting 

X Factor 

dFY 



I Bloalc I 


[ Weight I 


Freq 


L By. J 



Action 

OK 

CarKel 

Remove 

Recall 

Help 


O newy 


U. 


Bivariate 

Oneway 


L 

Logistic 

d 


Contingency 
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step 4: Click-on the “OK” button to generate the graph of FY by total cost of 
recruiting. 


Fit Y by X • Cootextuai ♦ JMP Pro 


IcdiB 


Bivahate 

iM 

Oneway 

u: 

Logistk 

Contingervcy 


Distribution of Y for e«ch X Modeling types d«terTntne enslysjs- 
Select Columns 
• 15 Columns 
^Run« 


4ltTotal Cost of Recnjibng 
^Advertising 

^Education Incentive 
^ota) Recruiters 
^jRecruiter Cost 
^CO 

Arp 

AfSOG 

Asci-mA 

Ajnemployment 

Atelative Pay 

AJMA_ 


Cast Selected Columns into Roles 
jy, Response ^Total Cost.. Recruiting 


X Factor , iL FY 


Block 

We»ghl 

Freq 


Remove 

Recall 

Help 


By 


Step 5: To add boxplots on the data for each FY, select the red triangle in the 
upper left hand corner of the graph. From the drop-down menu, select 
“Quantiles.” 

1^ JMP_129DesPts - Fit Y by X of Total Cost of Recruiting by FY - JMP Pro 


^^newav Analysis of Total Cost of Recruiting By FY 

Quantiles 

- Shows or hides a quantile report. | 

Means/Anova 

k • • 

Means and Std Dev 

a ! 1 

Analysis of Means Methods ► 

1 : • 

Compare Means ► 
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B. EXPLORE OUTLIERS FROM THE ONEWAY ANALYSIS GRAPH 



Step 1 : Hover your mouse over a data point of interest to retrieve information 
regarding that point. 


^ ^ Oneway Analysis of Total Cost of Recruiting By FY 

1000 


900 

800 

700 

600 

500 

400 

300 

200 


Row; 78 
FY: 2015 

Total Cost of Recruiting: 865.4414 


2015 


2016 


2017 


2018 

FY 


2019 


2020 


1 row selected 


2021 
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Clicking on the data point on the graph will highlight it within the greater data set. 
Understanding the input variables can help explain why the total cost of recruiting 
was unusually high for this data point. 


IJMP 1290ctPu 


^ r H p-» 


I » (“AlumntdS/t)) 

A Con et RM’njMHV} 

A Cdw<Mion InMAtivc 

A Rfouitm 

A Recruiter Co(t 

Axo 

Aif^ 

jHUXi 

.JTSCl-IBA 

A UnciT!)>iaym«it 

A Rdeove Pey 

JQMA 




Vlmed 

excluded 

Hidden 


Run 

• fV 
I 79 2015 
7* 2015 
90 2015 
81 2015 
92 2015 
Bi 2015 
94 2015 
85 2015 
99 2015 
B; 2015 

88 2015 

89 2015 

90 2015 

91 2015 
9? 201S 
9i 2015 

94 201S 

95 2015 

96 201S 

97 2015 
9* 2015 
99 2015 

IM 2015 
IftI 2015 


ti Recruiting 
965 Ml* 
25 7.21* 
959 1079 
271.5;rt» 
324 3377 
3i2.01i; 
420 076* 
i751M8 
4066709 
ii;*; 
899 4207 
284.4i;i 
585 5556 
*09^005 
*05 8755 
352.9529 
509 7756 
279.566; 
847 7716 
270.0264 
501 1516 
368 3U9 
528 4267 
itOI Mil 


Advertblng 

2799U 
16.6*96 
3211317 
25.0296 
201992 
25.7319 
64 2196 
484117 
774697 
20.1M1 
283 8923 
3D.(I3«6 
55 525 
22 7828 
57 8151 
38.3231 
1200012 
28.7556 
265 7515 
22.7017 
14 5 587 
37 2262 
167569 
158005 


EB 

3694925 
21.9775 
423 8939 
33.0391 
26 663 

33.9662 
84 7699 
63.9034 

102 26 
26.5374 
374 7378 
39.6457 
75 029 
30.0733 
76 5150 
50.5165 
158 4055 
37.9574 
5481517 

29.9663 
18 9271 
49138$ 
22 0926 

187 78S4 


2547 
2578 
2523 
2516 
3297 
3231 
3219 
3117 
2690 
2852 
2727 
2531 
5047 
3099 
5227 
3133 
27 U 
2501 
2789 
2563 
5180 
3352 
5445 
3336 


RetiuHa 
Cm 
209 6079 
211.1469 
206 6423 
206.0689 
270 0355 
264.8756 
263647 
255.2929 
219 SOU 
233.5885 
223 3506 
207.2975 
244 5596 
248.VOM 
2U1D25 
256.6013 
225 9259 
205.4U7 
228 4286 
209.9184 
260 4528 
274 5402 
282 157? 
273 2297 


NCO 
39531 
30859 
39699 
30231 
33125 
38516 
33203 
36563 
36953 
31406 
39453 
31175 
55515 
36172 
55484 
37ia 
40000 
32578 
54609 
3 3359 
50958 
36250 
50625 
38906 
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LRP HSK IIB8 

744 0 95 0.7 

}M 0.95 OJ 

744 0 95 07 

IM 0.95 07 


ReUive 
Piy QMA 
7 0 94375 199330* 
iX 1.04375 1883309 
5 9 0 978125 198330* 
5 1.03125 1883309 


III 7MP.1290nPU • M V by X o< lot^ Coil o< Keuuil>r>g by IV • JMP Pro 


iWoi 


d « OnewayArvBlysliofTotalCoitofRecruhingSyFY 

1000 




. 

Ron. 78 



rV: 2015 


tom Con o( RectutMig 8654414 | 



1 







1 






eveiuetionc done 


Step 2: To explore a group of outliers, lasso the data points of interest by 
creating a box around the data points with your mouse. Lassoing the data points 
will automatically select these data points within the greater data set. 
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step 3: The selected data points can be further analyzed on their own. Right- 
click on “Selected.” 


» Columns (15/1) 

^ Run * 

ilSS 

^ Total Cost of Recnjiting 

^ Advertising 

.JEB 

^ Education Incentive 

^ Total Recruiters 

^ Recruiter Cost 

^NCO 

^LRP 

^HSDG 

^TSCI-mA 

^ Unemployment 

^ Relative Pay 

^QMA 



Rows 

All roM 

qni 

Selected 

28 

Excluded 

0 

Hidden 

0 

Labelled 

0 


Then choose “Data View” from the drop down menu. 
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This will create a separate data table with just the outliers. 


File Edit Tables Rows Cols DOE Analyze Graph Tools View Window Help 




M Untitled 15 ^ 

< 1:^1 

Run # 

FY 

Total Cost of Recruitir>g 

Advertising 

EB 

Education 


iinJIredSubset This subset is linke 


Incentive 

T< 


1 

78 

2015 

865.4414 

279.911 

369.4825 


0 

- 


2 

80 

2015 

959.1079 

321.1317 

423.8939 


0 


!▼! Columns (15/0) 

3 

88 

2015 

889.4207 

283.893 

374.7378 


0 


^ Run # 

4 

96 

2015 

847.7716 

263.7513 

348.1517 


0 


AFY 

5 

207 

2016 

874.0407 

281.0028 

370.937 


0 


^ Total Cost of Recruiting 
^ Advertising 

JEB 

^ Education Incentive 
^ Total Recruiters 

6 

209 

2016 

968.0586 

322.3843 

425.5473 


0 


7 

217 

2016 

898.2175 

284.9996 

376.1995 


0 


8 

225 

2016 

856.4419 

264.7801 

349.5097 


0 

s 

9 

336 

2017 

879.807 

282.0989 

372.3705 


0 


^ Recruiter Cost 

10 

338 

2017 

974.1696 

323.6418 

427.2072 


0 


^NCO 
^LRP 
^HSDG 
^TSCI-niA 
^ Unemployment 

11 

346 

2017 

904.2436 

286.1113 

377.6669 


0 


12 

354 

2017 

862362 

265.8129 

350873 


0 


13 

465 

2018 

885.6706 

283.1992 

373.8229 


0 


14 

467 

2018 

980.3789 

324.9042 

428.8735 


0 


^ Relative Pay 

15 

475 

2018 

910.3705 

287.373 

379.14 


0 


^QMA 

16 

483 

2018 

868.3833 

266.8497 

352.2416 


0 

K1 Rows 

17 

594 

2019 

891.5424 

284.3038 

375.2811 


0 

All rows 28 

18 

596 

2019 

986.5974 

326.1715 

430.5464 


0 

Selected 28 

19 

604 

2019 

916.509 

288.3476 

380.6189 


0 

Excluded 0 

Hidden 0 

Labelled 0 

20 

612 

2019 

874.4168 

267.8906 

353.6155 


0 

21 

723 

2020 

897.9872 

285.4128 

376.7449 


0 



I "" “1 
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C. SELECT ONE FY TO ANALYZE 

To focus analysis on one specific FY, the other six FYs must be hidden and 
excluded. In this example, FY 2017 is the FY of interest. FYs 2015, 2016, 2018, 
2019, 2020, and 2021 will be hidden and excluded. 

Step 1: To exclude FY 2015 and 2016, select on the first row of FY 2015 data in 
the furthest column to the left. Hold the “shift” keyboard button. 


t5) JMP.129D«P1s - JMP Pro 







Fife £di( T«btet Rows 

Cots DO£ Analyze 

Oraph 

Toots View Window Help 

/ 

^ H -j 



JMP.129[>esPls 

> 




Total Cost of 







Recruit ng 

Advertising 




1 

2015 

302.7718 

25.2367 



2 

2 

2015 

364.2534 

50 3606 



- 3 — 

- 3— 

-»4§- 

407.1973 

84.0512 



4 

4 

2015 

272668 

15 5794 



5 

5 

2015 

314.7028 

24.5896 

iiFY 


6 

6 

2015 

455.8674 

80494 

^ ToUl Cost of Rccrurting 


7 

7 

2015 

296.7907 

9.7023 

A Advertising 


8 

8 

2015 

461.4439 

75.9782 

A £6 


9 

9 

2015 

386.4563 

66 0029 



10 

10 

2015 

410.5732 

77.775 










11 

11 

2015 

314.3641 

27.1973 

A NCO 


12 

12 

2015 

343.2203 

47,6491 

^LRP 


13 

13 

2015 

309.9515 

8.491 

^HSOG 


14 

14 

2015 

488.9989 

86.761 

A Unemployment 


15 

15 

2015 

309.0097 

7,2378 


16 

16 

2015 

423.4232 

56.3068 

AQMA 


17 

17 

2015 

424451 

84.3216 



18 

18 

2015 

377506 

61.8626 



19 

19 

2015 

299,8877 

27 8769 



20 

20 

2015 

324.3671 

34,8274 



21 

21 

2015 

331.2071 

22,3482 



22 

22 

2015 

427.W95 

69,7317 

V Rows 

23 

23 

2015 

290,6615 

6,5309 

All rows 903 

24 

24 

2015 

407.0701 

49,7876 



25 

25 

2015 

316279 

2887 


0 

26 

26 

2015 

422.3103 

74,2907 

Labelled 

0 

27 

27 

2015 

301.4469 

28.8314 




evaluations dor>e 
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step 2: Scroll down to the last row of FY 2016 data (which appears in row “258”). 
Click on the “258” cell in the furthest column to the left. 


Q JMP.1290esPts - JMP Pro 


He EM TeWcs Row 

cSI A - J 


Cob DOE AnilyTe Oraph Tools 


VteM W indow Hdp 


• ;MP.1290«Pts 


• CdunwHdS/l) 

ij JSLi 

^ Total Cost of Recruiting 

^ Ad««t>9ng 

iJEB 

Education IrKentive 
^ Total Recruiters 
^ Recruiter Cost 
JNCO 
^LRP 
JHSOG 
^TSCIEA 
^ Osemploytrient 
J RelaovePay 
JQMA 




Mm* 

FY 

Recruiting 

Ar^rertaing 

243 

2016 

307 3522 

9 6874 

244 

2016 

445 0398 

66.2517 

245 

2016 

303.2966 

12115 

246 

2016 

417.8577 

55 3918 

247 

2016 

341.4961 

38 6458 

248 

2016 

447 4629 

95 74U 

249 

2016 

348 5629 

394425 

250 

2016 

597.7795 

157.4634 

2S1 

2016 

299 0017 

1M2C6 

2S2 

2016 

393.1824 

430785 

2» 

2016 

345 3272 

33 8721 

254 

2016 

530.3328 

1144367 

255 

2016 

337 6762 

47 8482 

256 

2016 

289 7138 

166105 

—S*?- 

»» 1 

332 0182 

434822 

258 

2016 

335.8113 

38.4074 

259 

2017 

313 2553 

25434 

—260- 

2017 

37 5 2758 

507542 

261 

2017 

4179106 

84 7082 

262 

2017 

282.7752 

157012 

263 

2017 

325 5253 

24,7818 

264 

2017 

468 0033 

811232 


Step 3: Right-click on the selected rows, and choose “Hide and Exclude” from 
the drop down menu. 


Q JMP.1290esPU • JMP Pro 

File Fd« 7»ble Rem Cob OOE Andyze Graph Toob View Window Help 


c9 ^ M ^ 


JMP.129D«Pts 
« Source 


^ Cdunw (15/1) _ 

^FV 

Ji Total Cost of Recruiting 
^ Advertising 

A 16 

A Education bKentive 

A Total Recruiters 

A Recruiter Cost 

,dNCO 

JLRP 

JHSOG 

^TSCMHA 

A UnemploynTent 

A Relative Pay 

.dOMA 


♦ Rows 
Al rows 
Selected 
Excluded 
Hidden 



903 

2S8 

0 

0 


£ a ^ » /•. 



m 

FY 

Total Cost of 
Recruiting 

Advert bing 


i 248 

2016 

447,4629 

95,74U 


249 

2016 

348.5629 

39.4425 


250 

2016 

597.7795 

157.4634 

251 

251 

2016 

299.0017 

11.1203 

252 

1 252 

2016 

393.1824 

43.0785 

253 

253 

2016 

345.3272 

33,8721 

2it 

254 

2016 

530.3328 

1144367 

215 

255 

2016 

337.6762 

47.6482 

256 

256 

2016 

289.7138 

16.6105 

•' - -'IT-'- 


2016- 

-J32.0142 

43.4822 


Hide and Exclude 


txcKido'Unecclude 

HidcAInhide 

LdielAJnUbel 

Colon 

Markers 

Color Rows by Row State 
Select Matching CeNs 
Irwert Selection 
Clear Row States 
AddRows.. 

DeleteRows 


S8113 
J2SS3 
5 2758 
7.9106 
;2.77S2 
5 5253 
8.0033 
7.7777 
3.9173 
7.3943 
1.6413 
25i)67 
3 8085 
1 3336 
31.7J4 
0.4205 


38.4074 
25434 
50.7542 
84.7082 
15.7012 
24 7818 
81.1232 
9.7782 
76.5721 
665188 
78.3829 
27.4098 
48.0216 
8.5573 
87.4391 
7.2943 
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You should now see 
2016. 




next to each row of data from FY 2015 and 


EB JMP.129DesPts - JMP Pro 


File Edit Tables Rows Cob OOE Analyze Graph 

Toob 

Wew Window Help 

d ® d * -J 

--- ffl B- IL » 

/ 



» JMP.UPDejPa ^ 




Total Cost of 


^ Source 



1 

FY 

Recruiting 

Advert bmg 



248 

2016 

447.4629 

95.7411 


j'i' 


249 

2016 

348.5629 

39.4425 


®« i.;- 


250 

2016 

597.7795 

157.4634 

* Columns (lS/1) 

•ip* 


251 

2016 

299.0017 

111203 

^fy 



252 

2016 

393.1824 

43.0785 



253 

2016 

345.3272 

33.8721 

Total Cost of Recruiting 



254 

2016 

530.3328 

114.4367 

^ ArKertising 

®<rl 


255 

2016 

337.6762 

47.8482 

A E6 

^ Education Incentive 

®<» V'; 


256 

2016 

289.7138 

16.6105 

®« r. 


257 

2016 

332.0182 

43.4822 

A Recruiter Cost 



256 

2016 

335.8113 

38.4074 

^NCO 

259 

259 

2017 

313.2553 

25434 

^LRP 

JHSOG 

260 

260 

2017 

375.2758 

50.7542 

261 

261 

2017 

417.9106 

84.7082 

A Unemployment 
^ Rdatrve Pay 

262 

262 

2017 

282.7752 

15-7012 

263 

263 

2017 

325.5253 

24,7818 

MQMA 

264 

264 

2017 

468.0033 

81.1232 


Step 4: Follow steps 1-3 to hide and exclude data from FY 2018, 2019, 2020 
and 2021. Row 388 is the first row of data for FY 2018. 

ES JMP.U90«PU • JMP Pro 

Flic Edit Tables Rows Cob 0O€ Analyze Graph Toob View Window Hdp 




- JMP,1290esPts 

0 


is 


Total Cost of 


« Source 



FY 

Recruiting 

Advertbing 



377 

377 

2017 

451601 

96.1145 



378 

378 

2017 

352.6672 

39.5963 



379 

379 

2017 

602583 

158.0776 

» Columns (15/1) 

380 

380 

2017 

303092 

11,1637 

381 

381 

2017 

397.8625 

43.2465 

asd 

iFV 



382 

382 

2017 

34 9 5265 

34.0042 

^ Total Cost of Recruiting 


383 

383 

2017 

535.2327 

114.8831 

^ Advertisirig 

JEB 

^ Education Incentive 


384 

384 

2017 

34 1 4009 

48.0349 


385 

385 

2017 

293.5235 

16.6753 





335.7705 

436518 



^ Recruiter Cost 


387 

387 

2017 

339-7511 

38.5572 

JNCO 



388 

2018 

317.23 

25.5332 

JHSOG 


- ssr 

- 389— 

2018 ' 

37 9 5288 

50.9522 


390 

390 

2018 

421.9288 

85.0386 

^ Unemployment 
^ Relative Pay 


391 

391 

2018 

286.5432 

15,7624 


392 

392 

2018 

329 7009 

24.8785 

JQMA 


393 

393 

2018 

472 8652 

81.4396 



391 

394 

2018 

312.0732 

9.8163 



395 

395 

2018 

478085 

76.8708 



396 

396 

2018 

401.5732 

66.7783 



397 

397 

2018 

425.8783 

78.6886 



398 

396 

2018 

329.168 

27.5168 

» Rows 

399 

399 

2018 

357.8096 

48.2069 

All rows 

903 

400 

400 

2018 

325.8641 

8.5907 


258 

401 

401 

2018 

506.9392 

87.7802 

Hidden 

258 

402 

402 

2018 

324.9698 

7.3228 

Labelled 

0 

403 

4(B 

2016 

440.7446 

56.9683 


evaluations done 
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D. DISTRIBUTION 

The distribution of the total cost of recruiting for FY 2017 is explored. This 
technique can be applied to any of the output variables to better understand its 
distribution and possible spread values. 



Step 1 : Select “Analyze” from the ribbon, and select “Distribution” from the drop 
down menu. 


Ei JMP_129DesPts - JMP Pro 


File Edit Tables Rows Cols DOE i Analyze j Graph Tools View Window Help 


tB icJ 


^ Distribution 

JMP 129DesPts > 

< 

IL. FitYbyX 

Source 

W 

3 Matched Pairs 


o« 



o« 

Tabulate 



3a Fit Model 

^ Columns (15/1) 



A 

06 

Modeling * 

AFY 


Multivariate Methods ► 

A Total Cost of Recruiting 

06 


A Advertising 

06 

Quality and Process ► 


06 

Reliability and Survival * 

A Total Recruiters 

06 

A Recnjiter Cost 

06 

Consumer Research ► 


Distribution of a batch of values. 
Frequencies if categorical. Means and 
quantiles if continuous. Histograms, 
Box Plots, Quantile Plots. Tests on 
means. Fitting distributions. 
Capability. 


61.2543 

>1.0429 

58.6302 

>2.3941 

>2.9205 

38.0567 


16.9685 

113.5786 

13.87 

81.1356 

31.5034 

72.72 

53.2076 

74.0829 

9.8779 

67.5539 


22.3984 

149.9238 

18.3085 

107.099 

41.5845 

95.9904 

70.234 

97.7894 

13.0388 

89.1712 
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step 2: Select “Total Cost of Recruiting” from the list of columns, and click on the 
“Y, Columns” button. 


^ Distribution - jMP Pro 


The distribution of values in each column 
Select Columns 
^ 15 Columns 





Histograms Only 


Cast Selected 

Columns into Roles Action 

'Y, Columns 

required 



jUo'-' 

I Caned | 

[ Weight I 

Pfltr 

[ Remove | 

I 


I Recall ] 

I By I 


I Help j 


The distribution for Total Cost of Recruiting will appear. 

Step 3: To rotate the distribution to appear horizontal, click on the red triangle in 
the upper left hand corner of the graph, and select “Stack” from the drop down 
menu. 


& JMP_129DesPts - Distribution 


^•r^Distributions __ 

Uniform Scpltno 

Stock 

Arrange rn Ko>ss 

Save for Adobe Flash platform (5WF).~ 
Script 

700-1 • 

600 - 



Quantiles 

1001)% 

maximum 

974.1696 

99.5% 


974.1696 

97.5% 


875.44575 

90.0% 


533.85 

75.0% 

quartile 

433.12145 

50.0% 

median 

361.2135 

25.0% 

quartile 

320.18825 

10.0% 


293.5959 

2.5% 


276.82095 

0.5% 


261.6095 

0.0% 

minimum 

261.6095 


I 


/ 

% 


Rotates the histogram and stacks the 
g tfxlividual distnbution output 
1 vertical!,. 



0 

2021 

361.2343 

> 

1 

2021 

551.0429 

m 

2021 

338.6302 

583 

2021 

502.3941 


2021 

352.9205 

585 

2021 

438.0567 

586 

2021 

388 5754 

5C 

7 

2021 

424.45W 

588 

2021 

330.8024 

589 

2021 

471.6017 

590 

2021 

326.0588 

591 

2021 

443.76tM 

592 

2021 

363.5453 

!93 

2021 

469.9294 

594 

2oa 

371.0763 

J95 

20a 

623.6476 

596 

2oa 

321.5546 

597 

2021 

418.&U5 

598 

2021 

368.3852 

599 

2021 

556.9077 

KO 

2021 

358068 

VM 

lA'TI 

51A -rmc 
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E. PARTITION TREES 

The partition tree on total cost of recruiting will be explored. The partition tree is a 
useful method that can help provide insights into variable interactions. 



Step 1: To create a partition tree, select “Analyze” from the ribbon. Then choose 
“Modeling,” and “Partition” from the drop down menus. 


JMP_129C)esPts - JMP Pro 




^ JMP 129DesPt5 

> 

» Source 


▼ Columns (15/1) 



File Edit Tables Rows Cols DOE 

^ afll 


^ Analyze 


Graph Tools View Window Help 


AFY 

A Total Cost of Recruiting 
^ Advertising 

Education Incentive 
^ Total Recruiters 
^ Recruiter Cost 
^NCO 
JLRP 
^HSDG 
^TSCI-niA 
ji Unemployment 
^ Relative Pay 
^OMA 


Distribution 
FitYbyX 
Matched Pairs 


>■ Fit Model 


Modeling 


o« [T-n Tabulate 

o« 
o« 
o« 

OftL 

Ofi 
Ofi 
o« 

06 


Multivanate Methods 

Quality and Process 
Reliability and Survival 

Consumer Research 



' 889 

2U21 

Oa 890 

890 

2021 

oa 891 

891 

2021 

oa 892 

892 

2021 

OB 893 

893 

2021 


of 

9 

Advert isir>g 

Education 

EB IrKentive 

Total Recruiters 

Rec: 

52.0454 

90.5324 

119.5028 0 

2570 


a9.4008 

43.8552 

57.8889 0 

2633 


51.2343 

16.9685 

22.3984 0 

3461 


M.0429 

113.5786 

149.9238 0 

3078 



Partition 


► ^ Neural 

^ ^ Model Comparison 

► Nonlinear 

^ ^ Gaussian Process 

‘ Time Series 
\ Screening 
y Response Screening 


Recursively partition the data to 
predict a response. Classification and 
regression trees. 


2875 

2828 

2688 

3305 

3383 

3188 

3359 

2906 

2586 
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step 2: Select “Total Cost of Recruiting” from the list of columns, and click on the 
“Y, Response” button. 


Recursive partitioning 


Select Columns 
^ 15 Columns 


^Run ^ 

.^PV _, 

■■ ■ 1 ■ 1 

1 

^Advertising 




■^Education Incentive 


^otal Recruiters 


^Recruiter Cost 


.Jnco 


.Jlrp 


.Jhsdg 


.arsci-niA 


^Unemployment 


^Relative Pay 


^MA 



Cast Selected 

Columns into Roles 


Action 

Y, Response 

required 


1 OK 1 



1 Cancel | 

1 X, Factor | 

required 

1 Remove | 


1 Recall 1 

1 Help 1 

1 Weight | 



P ‘ 



1 1 

Optic-'- - ■■ 

1 Validation | 

—rv-' - -- - 

[ By j 

OptlO'-~. 





Informative Missing 
S Ordinal Restricts Order 
Validation Portion | 0 | 

Method [Decision Tree -| 


Step 3: Select each decision variable (Advertising, EB, Education Incentive, 
Total Recruiters) from the list of columns, then click on the “X, Factor” button. 


Recursive partitioning 
Select Columns 
^ 15 Columns 
^Run > 

&FY 



^ftecru Iter Cost 


.dlNCO 

.aiRP 

'.aHSDG 

j^TSCI-niA 

^Unemployment 

\^Relative Pay 

Aqma 

^ Informative Missing 
y Ordinal Restricts Order 
Validation Portion | 0 | 

Method [ Decision Tree -1 


Cast Selected Columns into Roles Action 


|y. Response] 

itf Total Cost... Recruiting 

X, Factor 

required 

1 Weight j 

opri''- > ■ 

F,„ 1 

_ -- 

1 Validation j 

. 

vyr- •• 

1 By 1 

cotiona: 
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step 4: Click on the “OK” button. 


Recureive partitioning 

Select Columns- 

^ IS Columns _ 

^Run * 

riiFY 

^Total Cost of Recruiting 

jr. 

AZ 

Al _ _ 

___ 

^Recnjiter Cost 
^^NCO 

Jlrp 

.Jhsdg 

^TSCI-mA 
^Unemployment 
^Relative Pay 
^MA 

[71 Informative Missing 

[7] Ordinal Restricts Order 

Validation Portion | 0 | 

I Decision Tree -| 


Cast Selected Columns into Roles Action 



Y, Response! 

^Total Cost... Recruiting 

X, Factor j 

^Advertising 

^Ee 

A Education Incentive 
^Total Recruiters 

Weight j 

J-pfior J: ■\.-:-i€r-Z 

I I 

optic - - c- 

Validation | 

o^:tiznairL’‘‘r'nc 

I I 

optiot'iol 




The partition tree window will pop-up with just the parent node. 


^ " Partitlonfor Total Cost of Recruiting 



^ All Rows 


Count 

129 

Mean 

398,24689 

StdDev 

124.63855 


> Candidates 

I T'B □ v 
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step 5: To make the first split on “Total Cost of Recruiting,” click on the “Split” 
button. 



Prune 1 1 Go 




Number 



RSquare 

RMSE 

N 

of SpKts 

AlCc 

Triining 

0.619 

76.624698 

129 

1 

1491.72 

Validation 

0.607 

78.407532 

774 




* All Rows 

Count 129 LogWorth Difference 

Me«n 398.24689 S7.40S143 38346 

StdDev 124.638SS 


Advert Hinge 139.1744 

* Advertlsing>-139.1744 

Count 120 

Count 9 

Mean 371.49385 

Mean 754.95403 

Std Dev 69.85107 

Std Dev 148.6531 

t> Candidates 

t> Candidates 


Continue to split, by clicking the “Split” button. If you want to undo a split, click on 
the “Prune” button. A “Training” value of 0.80 is an adequate threshold to 
achieve. In this case, disregard the “Validation” Revalue. 
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F. STEPWISE REGRESSION MODEL 

To develop a model for the total cost of recruiting, stepwise regression is used to 
determine the beta estimates to fit a model. 


Step 1 : Select “Analyze” from the ribbon, then “Fit Model” from the drop down 
menu. 


File Edit Tables Rows Cols DOE 


^ 1.: 

» JMP 129DesPts > 

< 

▼ Source 

w 


06 


06 



^ Columns (15/1) 

o«i 

A 

06 

iFY 

06 

^ Total Cost of Recruiting 

Ofi 

^ Advertising 

Ofi 



^ Education Incentive 

o« 

^ Total Recruiters 

A Recruiter Cost 

06 

A Kirn 



Window Help 


[!x FitYbyX 
^ Matched Pairs 

Tabulate 
Fit Model 
Modeling 

Multivariate Methods 

Quality and Process 
Reliability and Survival 

Consumer Research 


[of 

Advert isir>g 

B2.0454 

90.5324 

ft9.4008 

43.8552 


Linear models, including analysis of 
variance and multiple regression, 

{ variance components, Manova, 
stepwise regression, logistic 
regression, many more. 


-in 


J8.0567 
J8.5754 
M.4584 
JO,8024 

1 fiA17 


72.72 9 5.9904 

53.2076 70.234 

74.0829 97.7894 

9.8779 13.0388 

A7 ^^30 130 1717 


Step 2: Select “Total Cost of Recruiting” from the list of columns, and click on the 
“Y” button. 


A ^ Model Specification 

Select Columns 
^ 15 Columns 
i^Run 

^Education Incentive 
[^Total Recruiters 
^Recruiter Cost 
i,JNCO 
^LRP 

.Jhsog 

.jTSCI-niA 
^Unemployment 
.^Relative Pay 
|,JQMA 



Pick Role Variables 

required 


Freq 


By 


Weight ||cf: 


Personality: I 


Help 


Recall 


I Remove 


Run 


Keep dialog open 


Construct Model Effects 
I Add 

I Cross I 

[ Nest I 

I Macros ^ | 

Degree 
Attributes ▼ 

Transform ▼ 

1^ No Intercept 
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step 3: While holding the Ctrl key, select each market factor that was varied in 
the PROM-WED excursion. 


^ ▼ Model Specification 


Select Columns - 
15 Columns 


^Run^ 

An 


^Total Cost of Recruiting 
^Advertising 


^EB 

^Education Incentive 
^Total Recruiters 
^ Recru iter Cost 


.Jlrp 

^HSDG 

^SCI-mA 


A - 

Icz 

A}MA 


Pick Role Variables 


I V I 

^Total Cost of Recruiting 


I Weight | 

optional numeric 

I I 

optional numeric 

|validation| 

optional 


I llqpfton^ 


Personality: [standard Least Squares ^ | 
Emphasis: [Effect Leverage 

[ Help j [ Run ~] 

I Recall I [3 Keep dialog open 
[ Remove | 


Construct Model Effects 
Add 

[ Cross I 

[ Nest I 

[ Macros | 

Degree | 2 | 
Attributes 
Transform ▼ 
in No Intercept 


Select the right corner of the “Macros” button (i.e., the arrow), and select 
“Factorial to degree” from the drop-down menu. 


A - Model Specification 

Select Columns 
’’ 15 Columns 
^Run * 

^FY 

^otal Cost of Recruiting 

^Advertising 

AiZ 

^Education Incentive 
^Total Recruiters 
ai Recru iter Cost 

.Jlrp 
.Jhsdg 

.jTSCI-mA 
^MA 


lUnerr^lo^ment 


Pick Role Variables 


I V |l 

^Total Cost of Recruiting 

r‘'On.ol 

I 

I Weight I 


[ li 


I Validation! I 

'■ ^pticr-Q-. 

I By li 

ppftono/ I 


Construct Model Effects - 
I Add I 


Cross 



Factorial to degree 
factonal sorted 

Response Surface 


■ P*rso^*lity= [standard Least Squares 
Emphasis: [Effect Leverage 


[ Help ] f 

, [ Recall J [f1 Keep dialog open 


Remove 


Add selected columns and 
interactions up to the specified 
degree. E.g., degree 2 enters mam 
effects and two-way interactions. 
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This will add all main effect and two-way interactions. 


A Model Specification 

Select Columns 
^ 15 Columns _ 

^Run ^ 

^FY 

^Total Cost of Recruiting 

^Advertising 

^EB 

^Education Incentive 

^Total Recruiters 

^Recruiter Cost 

^NCO 

^LRP 

^HSDG 

^SCI-mA 

^Unemployment 

^Relative Pay 

^MA 


Pick Role Variables- 


I V I 

^Total Cost of Recruiting 


Optior-. 

I Weight | 

optional nun'^ric 

I Ffeq I 

optional r‘jr^ric 

I Validation! 

optional 


[ By ]I option^ 


Personality: |Standard Least Squares 
Emphasis; [ Effect Leverage 


[ Help I [ Run [ 

[ ] [r~| Kggp dialog open 

I Remove | 


Construct Model Effects 


i Add ] 

NCO 

i Cross I 

Unemployment 

Relative Pay 

Nest I 

NCO'Unemployment 

j Macros ▼ | 

NCO"Relative Pay 

Unemployment*Relative Pay 

Degree | 2 | 
Attributes ^ 

Transform ▼ 

n No Intercept 


Again, while holding the Ctrl key, select each market factor that was varied in the 
PROM-WED excursion. Select the right corner of the “Macros” button (i.e., the 
arrow), and select “Polynomial to degree” from the drop-down menu. This will 
add all second degree polynomial interactions. 


J « Model SpecrficAtion 

Select Columns 
^15 Columns 
a^Run ' 

^otal Cost of Recrurting 
^Advertising 

Education Incentive 
^oUl Recruiters 
^Recruiter Cost 

^ - - 

.^LRP 
^HSOG 
arsci-oiA 




.JQMA 


PKk Rote Variables 

Y I ^Total Cost of Recruiting 


Personality: , Suisdard Least Squares 


Emphasts. : Leverage 


I Werght | 

I ■ 

Validation 

Construct Model Effects 



Add 


NCO 

Unemployment 
Relative Pay 

1 ^ 4 ^ I NCO*Unemployment 
NCO’Relatrve Pay 


Cross 


^ I UnemploymenfRelative Pay 
Full Factorial 
Factorial to degree 
Factorial sorted 
Response Surface 
tiilntmeffas^cnia — 


Polynomial to Degree 
Schcffe CubK 


I Add the selected columns (sa^. X) 

and thew powers (X‘\ X’X'X. etc.) up 
to the specified degree. 
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step 4: From the “Personality” drop-down menu, select “Stepwise. 


A •• Model Specification 

Select Columns 
15 Columns 
^Run » 

Jfy 

^Total Cost of Recruiting 
^Advertising 

Jeb 

^Education Incentive 
^Total Recruiters 
^Recruiter Cost 

.Jnco 

^LRP 

.Jhsdg 

^rsci-niA 

^Unemployment 
^Relative Pay 


Pick Role Variables 



^Total Cost of Recruiting 

optiorz. 


I Weight | 


I freq I 

ostiora- : 

[validation! 

OptlO^Z- 


I By I _] 


Personality: 

Emphasis: 


I Help I 

I Recall I 

I Remove | 


[standard Least Squares 






Mixed Model 
Manova 

Loglinear Variance 


Nominal Logistic 
Ordinal Logistic 


Proportional Hazard 
Parametric Survival 


Construct Model Effects 


I Add 

[ Cross 

I Nest 

I Macros •* 


1 NCO 

Unemployment 
J Relative Pay 
"1 NCO*Unemployment 
^ NCO'Relative Pay 
J Unemployment'Relative Pay 


Generalized Linear Model 
Partial Least Squares 
Response Screening 


Degree 
Attributes ▼ 
Transform ▼ 


NCO'NCO 

UnemploymenfUnemployment 
Relative Pay'Relative Pay 


n No Intercept 


Step 5: Ensure that the “Keep dialog open” box is checked, and click the “Run” 
button. 


A '' Model Specification 

Select Columns- 

^ IS Columns _ 

^Run A 

Af'i 

^otal Cost of Recruiting 
^Advertising 

alEB 

^Education Incentive 

^Total Recruiters 

aiRecruiter Cost 

.^NCO 

ilLRP 

^HSDG 

.arse i-mA 

^Unemployment 

^Relative Pay 

Ajma 


Pick Role Variables 


( V I 

^Total Cost of Recruiting 


op* -iOi 

I Weight I 

- - 

[ Freq ] 

_rfl— r'lC 

I Validation I 



By 


Personality: [stepwise 


, Help 


Run 


Recall 

y Keep dialog open 


I Remove 



Construct Model Effects 


I l| 

NCO 

Unemployment 

Relative Pay 

NCO'Unemployment 

NCO'Relative Pay 

Unemployment'Relative Pay 

NCO'NCO 

UnemploymenfUnemployment 

Relative Pay'Relative Pay 

I Cross I 

I I: 

I Macros ▼ [ 

Degree | 2 | 
Attributes ^ 

Transform ^ 
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step 6: The “Stepwise Regression Control” window will appear. Press the “Go” 
button. 


^ ' Stepwise Fit for Totaf Cost of Recruiting 
^ Stepwise Regression Control 


Stopping Rulei [ Minimum BIC 
Direction: 

Rules; 


3 01 


Enter All i Meke Model 


I Forward 


[^11 Remove All] [ Run Model ] 


Combine 


(ZED 


77? used due to excluded rows or missing values. 



SSE 

DFE RMSE RSquare RSquareAdj 

Cp 

p AlCc 

BIC 



19884503 

128 124.63855 0.0000 0.0000 717.81915 

1 1614.135 

1619.76 

a Current Estimates 

Lock 

Entered Parameter 


Estimate 

nDF 

SS 

"F Ratio" 

■Prob>F" 



Intercept 


398.246889 

1 

0 

0.000 

1 

r, 


NCO 


0 

1 

7201112 

72.105 

4.6e-14 

r: 


Unemployment 


0 

1 

108286.7 

7.314 

0.00778 

O 

B 

RelaPve Pay 


0 

1 

533851.6 

46.610 

3.2e-10 

n 

D 

(NCO-35000.1)-(Unempioyment-60031) 


0 

3 

8415882 

30.576 

6.8e-15 

E 

C 

(NC0.35000,l)'(R*iative Pay-1) 


0 

3 

1459125 

114.857 

9.2e-36 

E 

D 

(Unemployment-6.0031)*(Reiative Pay*!) 


0 

3 

657070.1 

20.564 

6.8e-ll 

D 

D 

(NCO-35000.1)'(NCO-3500(1D 


0 

2 

8376808 

45.860 

lie-15 

D 

B 

(Unemployment-6.0031)*(Unemployment-6.0031) 

0 

2 

1109264 

3.722 

0.02688 

B 


(Relative Pay-l)'(ReiBtive Pay-1) 


0 

2 

5423772 

23.629 

1.93e-9 


Step 7: Once settled, select the “Run Model” button. 


^ . Stepwise Fit for Total Cost of Recruiting 
^ Stepwise Regression Control 


Stopping Rule: | Minimum BIC 

13] 

1^11 Enter All | 

'Melee Model 

Direction; [Forw.id -j 


[^] Remove All 

Run Model 


[Combine .] 

I Co l[ Slop II Step I 
774 rows not used due to excluded rows or missing values. 
SSE DFE RMSE RSquare RSquareAdj 

286974.74 120 48.902517 0.8557 0.8461 

J Current Estimates 


- Take current model and frt it in a separate full-featured fitting platform. 


Cp 

10.92502 


p AlCc BIC 
9 1382.197 1408.931 


Lock 

Entered 

Parameter 

Estimate 

nOF 

SS 

"F Ratio" - 

Prob>F' 



Intercept 

150,492117 

1 

0 

o.ox 

1 

Q 

E 

FKO 

0.02552672 

4 

1034475 

108.143 

7.7e-39 


V 

Unemployment 

•24.598171 

3 

132374.1 

18.451 

6.6e-10 

o 

E 

Relative Pay 

•550.24962 

4 

7341261 

76.745 

3.7e-32 

D 

E 

(NCO-35000.1)’(Unemployment-66031) 

-0,0035413 

1 

1606216 

6.716 

0.01074 

□ 

E 

(NCO-35000. l)*(Relative Pay-l) 

-0,1231431 

1 

163847.2 

68.514 

2e-13 


'j 

(Unemployment-6.0031)'(Relative Pay-l) 

84.0662X2 

1 

15660.78 

6.549 

0.01174 

E 

H 

(NCO-35000.l)*(NCO-35000.1) 

3.09283e6 

1 

6278413 

26.254 

1.16e-6 

n 

B 

(Unemployment-6.0031)'(Unemploynrtent-6.0031) 

0 

1 

6884615 

2.925 

0.08982 

r 

■! 

(Relative Pay-l)*(Relative Pay-1) 

1907.37931 

1 

6309642 

26.384 

l.le-6 

.4^ Step History 

Ste| 

Parameter Actkm 

"Sig Prob“ 

Seq SS RSejuare Cp 

p AlCc BIC __ 


1 (NCO-35000.1)*(RelativePay-D Entered 0.0000 1459125 0.7338 103.89 

2 Unemployment Entered 0.0000 105529.6 0.7869 61.055 

3 (NCO-35000.1)’(NCO-3500ai) Entered 0.0001 48541.67 0.8113 42432 

4 (Relative Pay-l)*(Relative Pay-1) Entered 0.0000 61580.54 0.8423 18.268 

5 (NCO-35000.1)’(Unemployment-6003D Entered 0.0377 1103826 0.8478 15.579 

6 (Unemployment-6.0031)*(RelativePayl) Entered 0.0117 15660.78 0.8557 10.925 

7 (Unemployment-6.0031)*(Unemployment-6.0031} Entered 0.0898 6884615 0.8591 10 

8 Best Specinc 0.8557 10.925 


4 1449S 146351 © 

5 142331 1439.78 C 

6 140986 142895C 

7 138981 141089 0 

8 1386.7 141093 O 

9 13822 140893 
10 138146 141086O 

9 13822 140893 ® 


E 
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The “Report: Fit Model” window will appear. 


9^ Report: Rt Model - JMP Pro [ g ( [s] 


A ▼ Fit Group 

A ▼ Response Total Cost of Recruiting 
A Effect Summaiy 

Source 
NCO 

Relative Pay 
NCO’Relative Pay 
Unemployment 
Relative Pay’Relative Pay 
NCO’NCO 

NCO’Unemployment 
Unemployment'Relative Pay 
Remove Add Edit B FDR 

^Summary of Fit 

RSquare 0.855679 

RSquareAdj 0,846058 

Root Mean Square Error 48.90252 

Mean of Response 398.2469 

Observations (or Sum Wgts) 129 

A Analysis of Variance 

Sum of 

Source DF Squares Mean Square 

Model 8 1701475.6 212684 

Error 120 286974,7 2391 

C Total 128 19 8 84 503 


1 Parameter Estimates 


Term 

Estimate Std Error t Ratio Prob>|t| 


Intercept 

150.49212 

68.20602 

2.21 0.0293* 


NCO 

0,0255267 

0.00148 

17.25 <.0001* 


Unemployment 

-24,59817 

3.699755 

-6.65 <.0001* 


Relative Pay 

-550.2498 

37.00036 

-14.87 <.0001* 


(NCO-35000.1)*(Unemployment-6,0031) 

•0,003541 

0.001366 

-2,59 0.0107* 


(NCO-35000.1)*(Relative Pay-1) 

-0,123143 

0.014877 

-8.28 <,0001* 


(Unemploy ment-6.0031)*(Relative Pay-1] 

84.0662 

32.8508 

2.56 0.0117* 


(NCO-35000.1)*(NCO-35000,1) 

3.0928e-6 

6,036e-7 

5.12 <.0001* 

r 

(Relative Pay-l)’(Relative Pay-1) 

1907,3793 

371335 

5.14 <.0001* 

4 


F Ratk) 
88.9351 
Prob > F 

<.ooor 



At this point, you can decide if you would like to make manual adjustments to the 
stepwise regression. For example, the interactions between unemployment rate 
and relative pay, and the new accession mission and unemployment in this 
example both exhibit low “t Ratio” values. 
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To remove these terms from the model, return to the “Stepwise Fit” window, and 
uncheck the terms in the “Entered” column that you would like to remove. Select 
“Run Model” to fit the new model. 


' Stepwise Fit for Total Cost of Recruiting 

^ Stepwise Regression Control 






Stopping Rule [ Minimum BIC ” ] EnttiAll 

Make Model 





Orecbon 

[forward »] [^| Remove AJI 

Run Model 





Rules: 

I Combine * 






I Go 

II Stop I Step 






774 rows not used due to excluded rows or missing values. 






$$E 

DEE RMSE RS<|uare RSquareAdj Cp 

p AlCc 

BIC 



303036d 

121 50.044352 0.8476 0.8388 15.749241 

8 1386.871 

1411.096 



A Current Estimates 

Lock Entered Parameter 

Estimate nOF 

ss 

"F Ratio" 

‘Prob>F- 


Intercept 151.602299 

1 

0 

0.000 

1 

0 

fKO 0.02552971 

3 

1018413 

135.548 

1.6e-38 


Unemployment 

•24.567887 

2 

116311.9 

23.221 

2.92e-9 


Relative Pay 

-550.43355 

4 

742311.5 

74.100 

1.3e-31 


(NCO*35000.1)*(Unemployment-61XJ3l) 

0 


1606246 

6.716 

0.01074 


(NCO-35000.1)'(ReldtivePayl) 

-0.1277552 

1 

1789104 

71.437 

7.6e-14 


(Unemployment'6.0031)*(RelabvePay>l) 68.0443421 

1 

1063688 

4.247 

0.04146 


(NCO-35000.1)*(NCO-35000D 

2.93986e^ 

1 

57275 

22.869 

4.94e6 


(Unemployment-6.0031)*(Unemployment-6.0031) 

0 

1 

9014824 

3.679 

0.05748 


(Relative Payl)‘(Relative Pay-l) 1917.11709 

1 

6374885 

25.454 

1.61e^ 

step History 


Step 

Parameter 

Action 

"SIg Prob" 

SeqSS 

RSquare 

Cp 

P 

AlCc 

BIC 

1 

(NCO-35000.1)’(ReJative Pay-1) 

Entered 

0.0000 

1459125 

0.7338 

103.89 

4 

14498 

146381 

2 

Unemployment 

Entered 

0.0000 

1055298 

0.7869 

61.055 

5 

142331 

1439.78 

3 

{NCO*35000.1)*(NCO-3500aD 

Entered 

0.0001 

4854187 

0.8113 

42.432 

6 

140986 

1428.95 

4 

(Relative Pay-l)*(Relative Pay-1) 

Entered 

0.0000 

6158054 

0.8423 

18.268 

7 

138981 

141089 

5 

{NCO-35000.1)*(Unemployment-60031) 

Entered 

0.0377 

1103826 

0.8478 

15.579 

8 

1386.7 

1410.93 

6 

(Unemployment-6.0031)*(Relat)ve Pay-1) 

Entered 

0.0117 

15660.78 

0.8557 

10.925 

9 

13822 

1408.93 

7 

(Unemployment-6.0031)"(UnemployrTTent-6.0031) Entered 

0.0898 

6884815 

0.8591 

10 

10 

138146 

141086 

8 

Best 

Specific 



0.8557 

10.925 

9 

13822 

140893 

9 

{NCO-35000.1)'(Unemployment-68031) 

Removed 

0.0107 

1606216 

0.8476 

15.749 

8 

138687 

14111 9 


Step 8: To graph the “Actual by Predicted” plot, select the red triangle next to 
“Response Total Cost of Recruiting.” From the drop-down menu, select “Row 
Diagnostics” and “Plot Actual by Predicted.” 


9^ Report Fit Model - JMP Pro 


^ Frt Group 

■ | - ij lcsDonse Total Cost of Recruiting 

Regression Reports 


Estimates 
Effect Screening 


pWorth 

33.553 El 
28.2731 


Row Diagnostics a 

^^-!—!-: 

Plot Actual by Predicted 

^av« (.ciumns a 

Model Dialog 

✓ Effect Summary 

Script a 

bticct Leverage 

Plot Residual by Predicted 

Plot Residual by Row 

Press 

Durbin Watson Test 



PVakie 

0.00000 

0.00000 

.AAQOftg.. 


AoBbCcDt 4oi 
^btle Enrv.. En 


Actual response value on Y axts, by 
Predicted value on the X axis. In good 
fits, points are near the dtagorral. You 
can see which points do not fit look 
for patterns, vrsualtze the test. 

-nn- 
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step 9: To fit the prediction model, select the red triangle next to “Response 
Total Cost of Recruiting.” From the drop-down menu, select “Estimates” then 
“Show Prediction Expression.” 


IV Report; Fit Model • JMP Pro 


A » Fit Group 


I - Response Total Cost of Recruiting 


Estimates » 

Show Prediction Expression 

tnect '>cfecning 

> 

licaeJ 

Factor Profiling 


Expanded Estimates 

Row Diagnostics 

> 

Sequential Tests 

Save Columns 

¥ 

Custom Test 

Model Dialog 

✓ Effect Summary 

Scnpt 

> 

MuKiple Comperisons 

Joint Factor Tests 

Inverse Prediction... 


Displ«/s or hides the prediction 
'ormule in the report. 

; u.vww 

0.00000 
0.00000 
0.00000 
0.00000 
0.00000 
0.01074 
0.01174 


181 































G. SCATTERPLOT MATRIX 

Scatterplot matrices can be used to visualize trends when multiple variables are 
changing. 



Step 1: Select “Graph” from the ribbon, then “Scatterplot Matrix” from the drop 
down menu. 


E§l JMP_129DesPts - JMP Pro 


File Edit Tables Rows 

Cols DOE Analyze 

d 3 ^ 


4 ^ Iti ffl II 

» JMP.129D«Pts 

> 

< 


▼ Source 






oa 878 




OA 879 




880 


^ Columns (15/1) 

OA 881 


.a ■■ -1 


on 882 


AFY 


oa 883 


^ Total Cost of Recruiting 

A Advertisinn 


oa 884 



kjraph 


Tools View 


Bubble Plot 


4^ Scatterplot Matrix 


Parallel Plot 
v:; Cell Plot 

Scatterplot 3D 
^ Contour Plot 
A Ternary Plot 


Window Help 


EducatM 


aA.nr^Ain.-CD- 

. Displays muttivanate data in a grid of 
^ 2-dimensional scatterplots. 




3 16.9685 

22.3984 


9 113.5786 

149.9238 


12 13.87 

18.3085 


1 81.1356 

107.099 


15 31.5034 

41.5845 


"tt "fi 

ne (MVt4 
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step 2: To set the Y-axis variables, select “Total Cost of Recruiting” and 
“Advertising” from the list of columns, and click on the “Y, Columns” button. 


Scatterplots of all pairs of Y variables, or all X-Y pairs if X's specified 



Action 



Matrix Format 


Lower Triangular » 


1'? 


Step 3: To set the X-axis variables, select the variables of interest (NCO, 
Unemployment Rate and Relative Pay in this case), and click on the “X” button. 


Scatterplots of all pairs of Y variables or all X-Y pairs if X’s specified 


Select Columns- 

15 Columns 
^Run * 

^FV 

I^Total Cost of Recruiting 
^Advertising 

.^Education Incentive 
^rfTotal Recruiters 

I rL.-' 

I^HSDG 
-.^SCI-niA 
{^Unemployment 
^Relative Pay 
.^MA 


Cast Selected Columns into Roles 


Y, Columns! 

Total Cost... Recruiting 
^Advertising 

jptic-. 

X 




Group 

OpflC"’J: 

( By I 

optior ; ■ 


Matrix Format [ Lower Triangular T 


Action 

OK 

I Cancel 


Remove 
[ Recall 
I Help 




Step 4: Repeat Step 3 for Unemployment Rate and Relative Pay. 

183 






















































































































step 5: To generate the scatterplot matrix, click the “OK” button. 


?!;)- Scatterplot Matrix - JMP Pro 


fU 


Scatterplots of all pairs of Y variables or all X-Y pairs if X's specified 


Select Columns 
15 Columns 
^jRun « 

iiFY 

^otal Cost of Recruiting 

^Advertising 

^EB 

^Education Incentive 
^otal Recruiters 
^Recruiter Cost 

.Jnco 

^LRP 

;.JHSDG 

^sci-niA 



Matrix Format 


Lower Triangular » 


Cast Selected Columns into Roles 


Y, Columns 

^Total Cost... Recruiting 
.^Advertising 


X 

.JNCO 

^Unemployment 

A Relative Pay 


Group 


By 




Action 



V □'T 


Step 6: To fit a trend line on the plots, click the red triangle, and select “Fit Line” 
from the drop down menu. 


/p^^atterolot Matrix 



300 

250 


.. V/ 

* • :•.* * 



• 


31000 34000 37000 

4 4555.5665775 

05 0.9 1 1.1 



Relative 

NCO 

Unemployment 

Pay 
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H. CONTOUR PLOTS 



Step 1: Select “Graph” from the ribbon, then “Contour Plot” from the drop down 
menu. 


E| JMP_129DesPts - JMP Pro 


File Edit Tables Rows Cols DOE Analyze 

Graph I 

[ Tools View Window 

Help 


I ^ Graph Builder I 

I 


JMP.129DesPts > 

< 


• Bubble Plot 




Educ 

Source 

w 

Ri 

ill), Scatterplot Matrix 



Advertising EB 

IiKe 


06i 878 


4 

90.5324 119.5028 



oa 879 


Parallel Plot 

18 

43.8552 57.8889 



Oa 880 


V;; Cell Plot 

13 

16.9685 22.3984 


Columns (15/1) 

Oa 881 



(9 

113.5786 149.9238 


aliiiSijy 

OA 882 


Scatterplot 3D 

12 

13.87 18.3085 


^fy 

883 


^ Contour Plot 


Displays three variables in a 2- 

^ Total Cost of Recruiting 

0<3 884 

1 

A Ternary Plot 

Y 

dimensional view where the third 

^ Advertising 

885 



variable is represented by contour 

JEB 

A rj __T_ Ml _ 

OS 886 


Surface Plot 


curves ui e^udi vuiuc. 

_ 1 
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step 2: To set “Total Cost of Recruiting” as the variable represented by the color 
scale, select “Total Cost of Recruiting” from the list of columns, and click the “Y” 
button. 


Please specify two X columns and one or more Y columns. 


Select Columns 
^ 15 Columns 


^A d v ertising 


Jeb 

^Education Incentive 

^Total Recruiters 

■^Recruiter Cost 

^NCO 

.Jlrp 

.Jhsdg 

.dtrsci-iHA 

^Unemployment 

^Relative Pay 

JQMA 

Options- 

Contour Values: 


Display: 

Data: 


Cast Selected Columns into Roles 


^Run > 

- 





L_ Fill Areas 
V Use Table Data 


Action 


I V I 

required rtumeric 



I X I 

required oumenc 
required rtumerk 


I By I 







OK 


Cancel 


Remove 

Recall 

Help 


Step 3: To set “Relative Pay” as the x-axis, select “Relative Pay” from the list of 
columns, and click the “X” button. 


Please specify two X columns and one or more Y columns. 

Select Columns r;ict Tnlumn^ infrn Rnl^ Artinn 




15 Columns 
^Run ^ 

Afy 

Jtr otal Cost of Recruiting 

^Advertising 

^EB 

^Education Incentive 
^Total Recruiters 
^Recruiter Cost 

.Jnco 

.Jlrp 

Jhsdg 

^TSCI-mA 


Options- 

Contour Values: | Specify... | 

Display: □ Fill Areas 

Data: [/ Use Table Data 

_ 
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step 4: To set the new accession mission (NCO) as the y-axis, select “NCO” 
from the list of columns, and click the “X” button. 



Step 5: Select the “Fill Areas” box, then click the “OK” button to generate the 
contour plot. 


Please specify two X columns and one or more Y columns. 


Select Columns 
^ 15 Columns 
^Run ^ 

4fy 

^Total Cost of Recruiting 
^Advertising 

^Education Incentive 
Jh'otal Recruiters 
^Recruiter Cost 

4 - 

Arp 
Asdg 
Asci-mA 
^Unemployment 
Aelative Pay 
AiMA 


Options- 

Contour Values: 


Specify... 


Display; 

Data: 


V Fill Areas 


y Use Table Data 


Cast Selected Columns into Roles 


I V I 

^Total Cost... Recruiting 


I ^ I 

^Relative Pay 
^NCO 


I By I 




Action 



□ ' 
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V. 


EXAMPLE TEST CASES 


Two test case examples are provided to demonstrate PROM-WED’s 
capabilities. 

A. EFFECT OF ECONOMIC UNCERTAINTIES 

What is the optimal allocation of recruiting resources that is robust to a 
broad range of economic uncertainties? 


Variable Type 

Variable Name 

Value Low 

Value Fligh 

Decision Variable 

Recruiters 

2,500 recruiters 

3,500 recruiters 

Market Factor 

Unemployment Rate 

4.0% 

8.0% 

Market Factor 

Relative Pay 

0.80 

1.20 

Policy Factor 

Recruiting Mission (NCO) 

30,000 recruits 

40,000 recruits 


B. EFFECT OF LEGALIZATION OF MARIJUANA TEST CASE: 

What is the optimal allocation of recruiting resources if the Navy desires to 
increase the percentage of high quality recruits from 70 percent to 85 percent? 
Due to uncertainties in the current fiscal environment, the unemployment rate 
may fluctuate between 4 to 8 percent and the ratio of relative pay may vary 
between 0.8 and 1.2. In addition, since marijuana has been legalized for 
recreational use in many states nationwide, drug-use amongst 18-24 year-olds is 
expected to increase. An increase in drug-use means less young adults qualify 
for military service. This test case models the effect of an annual decrease of 
10,000 qualified military available due to pre-service drug-use. 



Variable Name 

Value Low 


Decision Variable 

Production Recruiters 

2,500 

recruiters 

3,500 recruiters 

Market Factor 

Unemployment Rate (UE) 

4.0% 

8.0% 

Market Factor 

Percentage of Fligh Quality 

Recruits (TSC l-lll) 

70% 

85% 

Market Factor 

Relative Pay 

0.8 

1.2 

Market Factor 

Qualified Military Available 
(QMA) 

*See Table 13 

Policy Factor 

Recruiting Mission (NCQ) 

30,000 recruits 

40,000 recruits 
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Cumulative Effect of Decrease in QMA 


FY 

QMA Value Low 

QMA Value High 

2015 

1,873,304 

1,883,304 

2016 

1,863,304 

1,873,304 

2017 

1,853,304 

1,863,304 

2018 

1,843,304 

1,853,304 

2019 

1,833,304 

1,843,304 

2020 

1,823,304 

1,833,304 

2021 

1,813,304 

1,823,304 
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APPENDIX D. SCENARIO INPUT REPORTS 


A. PARAMETER INPUTS FOR FIGURE 45 

Where “Recruiters” is the only variable that is fixed. EB, NCF, and 


advertising are floated. 



2015 

2016 

2017 

2018 

2019 

2020 

2021 

NCF 

high 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

iow 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

UE 

high 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

iow 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

Recruiters 

high 

3913 

3913 

3913 

3913 

3913 

3913 

3913 

iow 

3913 

3913 

3913 

3913 

3913 

3913 

3913 

LRP 

high 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

iow 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

Advertising 

high 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

iow 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

EB 

high 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

iow 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

NCO 

high 

35025 

36425 

36800 

35800 

35225 

34650 

34650 

iow 

35025 

36425 

36800 

35800 

35225 

34650 

34650 

TSC 

high 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

iow 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

HSDG 

high 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

iow 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

Rei Pay 

high 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

iow 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

QMA 

high 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 

iow 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 
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B. PARAMETER INPUTS FOR FIGURE 46 


Where “Recruiters” is the only variable that is fixed. EB, NCF, and 
advertising are floated. 



2015 

2016 

2017 

2018 

2019 

2020 

2021 

NCF 

high 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

Low 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

UE 

high 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

Low 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

Recruiters 

high 

3500 

3500 

3500 

3500 

3500 

3500 

3500 

Low 

2500 

2500 

2500 

2500 

2500 

2500 

2500 

LRP 

high 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

Low 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

Advertising 

high 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

Low 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

EB 

high 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

Low 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

NCO 

high 

35025 

36425 

36800 

35800 

35225 

34650 

34650 

Low 

35025 

36425 

36800 

35800 

35225 

34650 

34650 

TSC 

high 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

Low 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

HSDG 

high 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

Low 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

Rei Pay 

high 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

Low 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

QMA 

high 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 

Low 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 
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C. PARAMETER INPUTS FOR TEST CASE 1 

Where “Recruiters” is the only variable that is fixed. EB, NCF, and 
advertising are floated. 



2015 

2016 

2017 

2018 

2019 

2020 

2021 

NCF 

high 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

Low 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

UE 

high 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

Low 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

Recruiters 

high 

3500 

3500 

3500 

3500 

3500 

3500 

3500 

Low 

2500 

2500 

2500 

2500 

2500 

2500 

2500 

LRP 

high 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

Low 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

Advertising 

high 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

Low 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

EB 

high 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

Low 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

NCO 

high 

40000 

40000 

40000 

40000 

40000 

40000 

40000 

Low 

30000 

30000 

30000 

30000 

30000 

30000 

30000 

TSC 

high 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

Low 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

HSDG 

high 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

Low 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

Rei Pay 

high 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

Low 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

QMA 

high 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 

Low 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 

1883304 
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D. PARAMETER INPUTS FOR TEST CASE 2 

Where “Recruiters” is the only variable that is fixed. EB, NCF, and 
advertising are floated. 



2015 

2016 

2017 

2018 

2019 

2020 

2021 

NCF 

high 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

Low 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

0.0001 

UE 

high 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

0.08 

Low 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

0.04 

Recruiters 

high 

3500 

3500 

3500 

3500 

3500 

3500 

3500 

Low 

2500 

2500 

2500 

2500 

2500 

2500 

2500 

LRP 

high 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

Low 

7.44 

11.22 

11.28 

11.38 

11.43 

11.46 

11.67 

Advertising 

high 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

Low 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

34.8264 

EB 

high 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

Low 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

40.971 

NCO 

high 

40000 

40000 

40000 

40000 

40000 

40000 

40000 

Low 

30000 

30000 

30000 

30000 

30000 

30000 

30000 

TSC 

high 

0.85 

0.85 

0.85 

0.85 

0.85 

0.85 

0.85 

Low 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

0.7 

HSDG 

high 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

Low 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

0.95 

Rei Pay 

high 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

1.2 

Low 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

QMA 

high 

1883304 

1873304 

1863304 

1853304 

1843304 

1833304 

1823304 

Low 

1873304 

1863304 

1853304 

1843304 

1833304 

1823304 

1813304 
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APPENDIX E. SCENARIO OF INTEREST TO N1 


The original baseline scenario request for a PROM-WED run was: 

For your baseline scenario we can use the 

• Current program of record for Recruiting mission is about 34000, so use: 
30000 - 40000 

• Advert: $60M so use $40M - $ 10OM 

• EB: $55M - SOM range 

• Recruiters (use Current onboard) I think they are at about 2900 so use 
2500 - 3500 

• Unemployment rate we use national rate and forecast per the Blue Chip 
Economic Indicators long range forecast. Which has current UE at -5.0% 
so use (4.0% - 8.0%>) 

• Vary relative pay between .8 and 1.2 

(Palmer, personal communication, 14 Sep 2016) _ 


Following continued communication with N1, the baseline scenario 
transitioned into a best case, worst case, and most likely case exploration. The 
following scenarios originated from that request. Test Case 1 and 2, explored 
within the report, combines all three of these cases into one PROM-WED run. 

A. BEST CASE 

The Navy’s best case scenario would be a low recruiting mission, no 
limitation on the number of recruiters in the field, and favorable economic 
conditions for recruiting (i.e., high unemployment rate and relative pay favoring 
the military versus the civilian sector). Table 17 shows the variables that this 
scenario focuses on. In this case, all decision variables will be optimized. 


Table 17. Scenario of Interest: Best Case 


Variable Type 

Variable Name 

Value Low 

Value High 

Decision Variable 

Recruiters 

Float 

Market Factor 

Recruiting Mission (NCO) 

30,00( 

D recruits 

Market Factor 

Relative Pay 

1.00 

1.20 

Market Factor 

Unemployment Rate 

5.5% 

8.0% 
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B. WORST CASE 


The Navy’s worst case scenario would be a high recruiting mission, a 
limited number of recruiters in the field, and an economic environment that is 
unfavorable to recruiting (i.e., the unemployment rate is low and the relative pay 
favors the civilian sector). The inputs for the worst case scenario are shown in 
Table 18. In this case, the number of recruiters is fixed and all other decision 
variables will be optimized. 


Table 18. Scenario of Interest: Worst Case 


Variable Type 

Variable Name 

Value Low 

Value High 

Decision Variable 

Recruiters 

2,500 recruiters 

Market Factor 

Recruiting Mission (NCO) 

40,000 recruits 

Market Factor 

Relative Pay 

0.80 

1.00 

Market Factor 

Unemployment Rate 

4.0% 

5.5% 


C. MOST LIKELY 

The most likely scenario that the Navy will face is a moderate recruiting 
mission, a limited range of available recruiters, and a balanced economic 
situation that naturally fluctuates between favorable and unfavorable conditions 
for recruiting. Table 19 shows the input variables for this scenario, where number 
of recruiters is fixed and tested over a range of values. All other decision 
variables will be optimized. 


Table 19. Scenario of Interest: Most Likely 


Variabie Type 

Variable Name 

Value Low 

Value High 

Decision Variable 

Recruiters 

2,500 recruiters 

3,000 

recruiters 

Market Factor 

Recruiting Mission (NCO) 

35,000 recruits 

Market Factor 

Relative Pay 

0.80 

1.20 

Market Factor 

Unemployment Rate 

4.5% 

6.5% 
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APPENDIX F. DISTRIBUTIONS 


A. FY 2017 DISTRIBUTIONS FOR TEST CASE 1 


'' Distributions 


^ •• Advertising 


hd^]-. 





---- 1 -1-T-f^-[=r-- 

0 50 100 1 50 200 2 50 300 350 


^ - EB 



Quantiles 


^ '' Summary Statistics 

100.0% 

maximum 

323.6418 

Mean 

58.118597 

99.5% 


323.6418 

Std Dev 

56.188302 

97.5% 


278.0274 

Std Err Mean 

4.9471041 

90.0% 


120.9422 

Upper 95% Mean 

67.907288 

75.0% 

quartile 

73.9048 

Lower 95% Mean 

48.329906 

50.0% 

median 

39.7348 

N 

129 

25.0% 

quartile 

22.92 



10.0% 


13.8271 



2.5% 


7.61005 



0.5% 


6.5819 



0.0% 

minimum 

6.5819 




Quantiles 


^ '' Summary Statistics 

100.0% 

maximum 

427.2072 

Mean 

76.716542 

99.5% 


427.2072 

Std Dev 

74.168552 

97.5% 


366.996125 

Std Err Mean 

6.5301769 

90.0% 


159.6437 

Upper 95% Mean 

89.637612 

75.0% 

quartile 

97.5543 

Lower 95% Mean 

63.795471 

50.0% 

median 

52.4499 

N 

129 

25.0% 

quartile 

30.2544 



10.0% 


18.2518 



2.5% 


10.0453 



0.5% 


8.6882 



0.0% 

minimum 

8.6882 
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B 


FY 2017 DISTRIBUTIONS FOR TEST CASE 2 


Distributions 


A Advertising 


- 1 . 





-r— — — — —’—T " T—’——T— 

0 50 100 150 200 250 300 350 


4 - EB 



Quantiles 

A 

» Summary Statistics 

100.0% maximum 

342.3389 

Mean 

71.101741 

99.5% 

342.3389 

Std Dev 

60.307798 

97.5% 

278.834575 

Std Err Mean 

5.3098056 

90.0% 

131.319 

Upper 95% Mean 

81.608099 

75.0% quartile 

86.7565 

Lower 95% Mean 

60.595384 

50.0% median 

50.8824 

N 

129 

25.0% quartile 

31.68145 



10.0% 

20.0803 



2.5% 

10.519725 



0.5% 

6.7331 



0.0% minimum 

6.7331 




Quantiles 

A 

'' Summary Statistics 

100.0% maximum 

451.8873 

Mean 

93.854303 

99.5% 

451.8873 

Std Dev 

79.606289 

97.5% 

368.061575 

Std Err Mean 

7.0089429 

90.0% 

173.341 

Upper 95% Mean 

107.72269 

75.0% quartile 

114.5186 

Lower 95% Mean 

79.985912 

50.0% median 

67.1648 

N 

129 

25.0% quartile 

41.8195 



10.0% 

26.506 



2.5% 

13.88605 



0.5% 

8.8877 



0.0% minimum 

8.8877 
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